reliable acceptance testing
Post on 11-May-2015
1.378 Views
Preview:
TRANSCRIPT
Dagfinn Reiersøl, ABC Startsiden 1
Reliable acceptance testing
Dagfinn Reiersøl, ABC Startsiden 2
Manual testing is immoral
• Not what humans are good at
• Time consuming = expensive
• High stress
• Error-prone
• Exception: exploratory testing
Dagfinn Reiersøl, ABC Startsiden 3
Types of automated testing
• Unit tests: clean code
• Acceptance tests: communicate with user / customer
• Other functional tests: ensure that everything works
Dagfinn Reiersøl, ABC Startsiden 4
High-level challenges of acceptance testing
• Getting the Product Owner involved
• Test Doubles
• Not over-testing the user interface
Dagfinn Reiersøl, ABC Startsiden 5
Unit tests
• Keep code clean
• Avoid “hidden” bugs
• Aid design
• Test-driven development
Dagfinn Reiersøl, ABC Startsiden 6
Scrum Product Owner
• Represents customer(s) and other stakeholders
• Owns the product backlog
• Is not a developer
Dagfinn Reiersøl, ABC Startsiden 7
Acceptance tests
• For the Product Owner or possibly testers
• Goal: Make sure we implement the right features
• Ideally editable by a non-programmer
• Tests features, not modules
• Make sure you're done
Dagfinn Reiersøl, ABC Startsiden 8
Other functional tests
• Goal: Make sure the implementation works
• Could be testing the same thing as acceptance tests, but
• need not be business-readable
Dagfinn Reiersøl, ABC Startsiden 9
The Test Double pattern
Dagfinn Reiersøl, ABC Startsiden 10
Surrounded by “Test Doubles”
Dagfinn Reiersøl, ABC Startsiden 11
Web testing: advantages
• You can test exactly what the user sees
• There are tools to test JavaScript-heavy / Ajax apps
• There are tools to record and play back tests
• “It seemed like an good idea at the time”
Dagfinn Reiersøl, ABC Startsiden 12
Web testing: disadvantages
• Fragile: Small UI changes break tests
• Fragile: External services break tests
• Slow
• Harder to replace external services with fakes
Dagfinn Reiersøl, ABC Startsiden 13
Strategies for robustness
• Test Bus
• Test patterns
• Test refactoring
• Making tests unnecessary
Dagfinn Reiersøl, ABC Startsiden 14
Web testing: essentials
• Must be in a programming language (conditionals, variables)
• Language not necessarily PHP, since HTTP allows communication between languages
• “Business-readable” programming language (FitNesse wiki, Gherkin...)
Dagfinn Reiersøl, ABC Startsiden 15
Web testing tips
• Test the HTML output using regex, XPath, CSS selector
• Use element IDs or names to test links, forms and fields
• Log HTTP requests in the application
Dagfinn Reiersøl, ABC Startsiden 16
How to use the request log
Automated test fails but not manual run from browser
• Run the automated test, find the failing request in the log.
• Run the app manually, find the successful request in the log.
• Compare.
Dagfinn Reiersøl, ABC Startsiden 17
Using SimpleTest
<?phprequire_once 'simpletest/autorun.php';require_once 'simpletest/web_tester.php';error_reporting (E_ALL);
class JoomlaSimpleTestTest extends WebTestCase {
function testJoomla() { $this->get('http://localhost/joomla'); $this->assertResponse(200); $this->assertTitle('Welcome to the Frontpage'); $this->assertText('Welcome to the Frontpage'); $this->click('Welcome to Joomla!'); $this->assertPattern('/Joomla! is a free.*content
publishing system/is'); }}
Dagfinn Reiersøl, ABC Startsiden 18
Using PHPUnit with Selenium
<?phprequire_once 'PHPUnit/Extensions/SeleniumTestCase.php';class JoomlaPhpUnitTest extends
PHPUnit_Extensions_SeleniumTestCase { protected function setUp(){ $this->setBrowser('*firefox'); $this->setBrowserUrl('http://localhost/'); } public function testJoomla() { $this->open('http://localhost/joomla/'); $this->assertTitle('Welcome to the Frontpage'); $this->assertTextPresent('Welcome to the Frontpage'); $this ->clickAndWait( '//ul[@class=\'latestnews\']/li[4]/a'); $this ->assertTextPresent('Joomla! is a free open'); }}
Dagfinn Reiersøl, ABC Startsiden 19
SimpleTest vs. PHPUnit / Selenium
Simpletest:
– Faster
– Easier to use
PHPUnit with Selenium
– Heavier but more advanced
– Can test JavaScript-heavy / Ajax apps
– Can test cross-browser compatibility
Dagfinn Reiersøl, ABC Startsiden 20
Don't overdo Selenium
“He ate a lot of brazil nuts which is a big deal because they contain selenium, which in high doses causes fatigue, vomiting, skin irritation, discharge from the fingernail beds and hair loss.”
- House, M.D.
Dagfinn Reiersøl, ABC Startsiden 21
Acceptance test frameworks
• FitNesse: has specific support for PHP
• Cucumber: has PHP-related documentation
• Robot Framework
• Concordion
Dagfinn Reiersøl, ABC Startsiden 22
Using FitNesse with SimpleTest
Dagfinn Reiersøl, ABC Startsiden 23
PhpSlim Fixture for FitNesse test (simplistic!)
<?phprequire_once 'simpletest/browser.php';
class SimpleTestFixture { function __construct() { $this->browser = new SimpleBrowser; }
function goToPage($url) { $this->content = $this->browser->get($url); return !empty($this->content); }
function pattern($pattern) { return preg_match("/$pattern/is",$this->content) > 0; }}
Dagfinn Reiersøl, ABC Startsiden 24
Using Cucumber with Webrat
• joomla.feature (Cucumber Gherkin business-readable)
• webrat_steps (Ruby code testing via HTTP)
Feature: Joomla welcome In order to understand what Joomla is As a potential user I want to be able to read an introduction to Joomla
Scenario: Welcome page Given I am on the home page When I follow "Welcome to Joomla!" Then I should see "Joomla! is a"
Dagfinn Reiersøl, ABC Startsiden 25
Webrat steps for Cucumber
• Ruby code runs Webrat which runs the web app using HTTP
Given /^I am on (.+)$/ do |page_name| visit path_to(page_name)end
When /^I follow "([^\"]*)"$/ do |link| click_link (link)end
Then /^I should see "([^\"]*)"$/ do |text| response_body.should contain(text)end
Dagfinn Reiersøl, ABC Startsiden 26
Avoiding difficult tests
• It's hard to test behaviors that cut across the HTTP request
• Improve architecture so these behaviors can't fail
• Example: duplication between input control name and HTTP variable name
Dagfinn Reiersøl, ABC Startsiden 27
Hard to test: name duplication
<input type="text" name="address1" value="<?= $address1 ?>">
// Oops...
echo $_POST['address_1'];
Dagfinn Reiersøl, ABC Startsiden 28
Getting rid of name duplication
• Use a form handling package
• Or do it yourself. Simplistic version here:
<?php define('ADDRESS1_NAME', 'address1');$address1_value = $_POST[ADDRESS1_NAME];
<html>...<input name="<?= ADDRESS1_NAME ?>" value="<?=
$address1_value ?>">
Dagfinn Reiersøl, ABC Startsiden 29
Refactoring tests
• Refactor or be doomed to drown in duplicate code
• Especially true of web tests
• Most important: eliminate duplication
Dagfinn Reiersøl, ABC Startsiden 30
BDD-style test names
class JoomlaSimpleTestTest extends WebTestCase {
function setUp() { $this->get('http://localhost/joomla'); }
function testFrontPageIsValid() { $this->assertResponse(200); $this->assertTitle('Welcome to the Frontpage'); $this->assertText('Welcome to the Frontpage'); }
function testWelcomePageIsValid() { $this->click('Welcome to Joomla!'); $this->assertPattern('/Joomla! is a free.*content
publishing system/is'); }}
Dagfinn Reiersøl, ABC Startsiden 31
Using custom assertions instead
class JoomlaSimpleTestTest extends WebTestCase { function testJoomla() { $this->get('http://localhost/joomla'); $this->assertValidHomePage(); $this->click('Welcome to Joomla!'); $this->assertValidWelcomePage(); } function assertValidHomePage() { $msg = 'Error on home page'; $this->assertResponse(200,$msg); $this->assertTitle('Welcome to the Frontpage',$msg); $this->assertText('Welcome to the Fontpage',$msg); } function assertValidWelcomePage() { $this->assertPattern( '/Joomla! is a free.*content publishing system/is', 'Error on welcome page'); }
Dagfinn Reiersøl, ABC Startsiden 32
ComposedRegex
http://martinfowler.com/bliki/ComposedRegex.html
• This regex tolerates markup (or anything else) between the content items
• Yes, I know the email regex is simplistic
$email = '.*\w+@\w+\.\w+.*'; $firstname = '.*\w.*'; $lastname = '.*\w.*'; $date = '.*20\d\d-\d\d-\d\d( | |T)\d\d:\d\d.*'; $regex = 'Email address:'. $email . 'Name: ' . $firstname . $lastname . 'Date:' . $date;
Dagfinn Reiersøl, ABC Startsiden 33
Re-using setup
• Refactor gradually as needed:
– When test class gets big, split it and...
– ...extract setup method into a separate class
– When setup is common to many tests...
– ...extract setup class into a separate file
class JoomlaTestSetup { function setUp() { $this->get('http://localhost/joomla'); }}
class JoomlaSimpleTestTest extends JoomlaTestSetup..
Dagfinn Reiersøl, ABC Startsiden 34
Useful types of classes in web testing
• Scraper
• Domain Object
• Specification
Dagfinn Reiersøl, ABC Startsiden 35
Domain object test version
class UserTest extends WebTestCase { function setUp() { $this->user = new TestUser( 'Jane','Doe','jane@example.com'); }
function testCanRegisterUser... $this->setFieldByName( 'firstname', $this->user->firstname );
Dagfinn Reiersøl, ABC Startsiden 36
Given-when-then in PHP
class JoomlaSimpleTestTest extends WebTestCase { function setUp() { $this->pages = array( 'JoomlaHomePage' => 'http://localhost/joomla'); } function testWelcomPageIsValid() { $this->givenIAmOn('JoomlaHomePage'); $this->whenIFollow('Welcome to Joomla!'); $this->thenIShouldSee('Joomla! is a free'); } function givenIAmOn($name) { $this->get($this->pages[$name]); } function whenIFollow($text) { $this->clickLink($text); } function thenIShouldSee($pattern) { $this->assertPattern("/$pattern/is"); }
Dagfinn Reiersøl, ABC Startsiden 37
Test bus
http://www.martinfowler.com/ieeeSoftware/testBus.pdf
Dagfinn Reiersøl, ABC Startsiden 38
Recommendations from Uncle Bob
• Test the features through the test bus
• Test the user interface separately in isolation
• Run a few end-to-end tests to test the “plumbing”
Dagfinn Reiersøl, ABC Startsiden 39
Test bus
Where is the test bus? Some possbilities:
• Test the Model
• Test the Controller
• Test the server side of an Ajax or “thin server” application
• Test a public API
Dagfinn Reiersøl, ABC Startsiden 40
Test bus with presentation tests
Dagfinn Reiersøl, ABC Startsiden 41
Fake web service
• Easier to do when not testing via HTTP
• Sensing/verification is very hard to do
Dagfinn Reiersøl, ABC Startsiden 42
Self-initializing fake
http://martinfowler.com/bliki/SelfInitializingFake.html
Dagfinn Reiersøl, ABC Startsiden 43
Self-initializing fake
http://martinfowler.com/bliki/SelfInitializingFake.html
top related