Download - Plugin Testing
Plugin Testing
Thursday, October 22, 2009
Plugin Testing
Thursday, October 22, 2009
Plugin TestingPlugins & other stu
ff
Thursday, October 22, 2009
Why Test?
• Correctness
• Performance
• Knowing Your Code
• Documentation
Thursday, October 22, 2009
Cost of Change
v1.0 v2.0 v3.0 v4.0
Tim
e
Development Feature Testing Regression Testing
Thursday, October 22, 2009
Feedback Loops
Development
Testing
Testing
JIRA
Thursday, October 22, 2009
Feedback LoopsDevelopment
Testing
Testing
JIRA
Thursday, October 22, 2009
Feedback LoopsDevelopment
Testing
Testing
JIRA
Thursday, October 22, 2009
Cost of Change
v1.0 v2.0 v3.0 v4.0
Tim
e
Development Writing Tests Executing Tests
Thursday, October 22, 2009
Eat Your VegetablesThursday, October 22, 2009
Variety
http://www.flickr.com/photos/sharontroy/3606495880/Thursday, October 22, 2009
Ingredients Matter
http://www.flickr.com/photos/sharontroy/3938831210/Thursday, October 22, 2009
Ingredients + Technique = Art
http://www.flickr.com/photos/sharontroy/3606264108/Thursday, October 22, 2009
No Regretshttp://www.redflagdeals.com/forums/costco-west-deals-dove-nailer-lays-tuna-gain-pantene-734644/Thursday, October 22, 2009
Test-Driven Developmenthttp://www.agiledata.org/essays/tdd.htmlThursday, October 22, 2009
Types of Tests
Unit Integration
Thursday, October 22, 2009
Unit Tests
Thursday, October 22, 2009
What is Unit Testing?
• Tests individual components
• Avoids state & external systems
• Runs in development environment
Thursday, October 22, 2009
Why Unit Test?
• Correctness
• Improve Design
• Design by Contract
Thursday, October 22, 2009
How to Unit Test
• Set up
• Execute
• Verify
• Clean up
Thursday, October 22, 2009
How to Unit Test an Atlassian Plugin
Thursday, October 22, 2009
Unit Testing Tools
Thursday, October 22, 2009
JUnit
public class UriTest { @Test public void absoluteUriParamIsReturned() { URI absoluteUri = URI.create("http://www.example.com"); URI resolvedUri = resolveUriAgainstBase( "http://localhost:8080", absoluteUri) assertEquals(absoluteUri, resolvedUri); }}
Thursday, October 22, 2009
JUnit
@Test(expected= GadgetSpecUriNotAllowedException.class)public void blankLocationThrowsException() { urlBuilder.parseGadgetSpecUrl( BASE_GADGET_SPEC_PATH + PLUGIN_KEY + "/" + "");}
Thursday, October 22, 2009
MockitoDashboard dashboard = mock(Dashboard.class);DashboardStore store = mock(DashboardStore.class);when(dashboard.getId()) .thenReturn(DASHBOARD_ID);when(store.update(dashboard)) .thenReturn(DashboardStore.SUCCESS);
repository.save(dashboard);
verify(store) .update(DASHBOARD_ID, dashboard);verifyNoMoreInteractions(store);
Thursday, October 22, 2009
Hamcrest
Iterable<ColumnIndex> range = ColumnIndex.range(ColumnIndex.ZERO, ColumnIndex.ONE);Iterator<ColumnIndex> columnIt = range.iterator();assertThat(columnIt.next(), is(equalTo(ColumnIndex.ZERO)));assertThat(columnIt.next(), is(equalTo(ColumnIndex.ONE)));assertFalse(columnIt.hasNext());
Thursday, October 22, 2009
Hamcrest
assertThat(iterator.next().getState(), is(sameInstance(gadget.getState())));
assertThat(store.entries(), hasItem(GADGET_SPEC_URI));
assertThat(staticContentTab.asText(), containsString("Static Content"));
assertThat(changesList.get(1), is(deeplyEqualTo( new RemoveGadgetChange(gadgetId))));
Thursday, October 22, 2009
Clover
Thursday, October 22, 2009
Unit Testing Traps
• False Positives
• False Negatives
• Flapping
Thursday, October 22, 2009
Non-Test
• Doesn’t test what it appears to test
• No assertion
• Passes when code is broken
• Solutions:
• Test-Driven Development
• IDE inspections, PMD, FindBugs
Thursday, October 22, 2009
Missing Test
• Doesn’t test full range of inputs
• Boundary conditions, special cases
• Passes when code is broken
• Solutions:
• Clover
• JUnit Theories
Thursday, October 22, 2009
Coverage as a Crutch
• Over-reliance on Clover
• Misses code paths, boundary cases
• Passes when code is broken
• Solutions:
• Your brain
• Peer review
Thursday, October 22, 2009
Side-Effects
• Produces or relies on side-effects
• Maintains state between runs
• Fails inconsistently
• Solutions:
• Mock objects
• @Before/@After
Thursday, October 22, 2009
Over Constraint
• Tests implementation details
• Overuse of mocks
• Fails when code is not broken
• Solutions:
• Stubs/fakes
• Refactoring
Thursday, October 22, 2009
Unit Testing Traps
• False Positives
• False Negatives
• Flapping
←More tests
←Reduce coupling
←Isolate
Thursday, October 22, 2009
Integration Tests
Thursday, October 22, 2009
What is Integration Testing?
• Tests full system
• Tests each supported configuration
• Runs in simulated production environment
• Many types: functional, load, stress, etc.
Thursday, October 22, 2009
Why Integration Test?
• Correctness
• Robustness
• Performance
Thursday, October 22, 2009
• Set up
• Execute
• Verify
• Clean up
How to Integration Test
Thursday, October 22, 2009
How to Integration Test a Web Application
• Deploy your plugin or application
• Load data
• Simulate or script a web browser
• Parse the HTML to verify results
• Clean up
Thursday, October 22, 2009
How to Integration Test anAtlassian Plugin
Thursday, October 22, 2009
Integration Testing Tools
Thursday, October 22, 2009
JWebUnit
beginAt("/home");clickLink("login");assertTitleEquals("Login");setTextField("username", "test");setTextField("password", "test123");submit();assertTitleEquals("Welcome, test!");
Thursday, October 22, 2009
HtmlUnit
WebClient webClient = new WebClient();HtmlPage page = webClient.getPage("http://example.net");HtmlDivision div = page.getHtmlElementById("some_div_id");HtmlAnchor anchor = page.getAnchorByName("anchor_name");List<?> divs = page.getByXPath("//div");HtmlDivision div = page.getByXPath("//div[@name='John']") .get(0);
Thursday, October 22, 2009
JIRA/Confluence Test Frameworks
PageHelper helper = getPageHelper();
helper.setSpaceKey(spaceKey);helper.setParentId(parentId);helper.setTitle(title);helper.setContent(content);helper.setCreationDate(new Date());helper.setLabels(labels);assertTrue(helper.create());
Thursday, October 22, 2009
JIRA/Confluence Test Frameworks
addUser("john.doe");addUserToGroup("john.doe", JIRA_DEV);activateTimeTracking();String issueKey = addIssue(TEST_PROJECT_NAME, TEST_PROJECT_KEY, "Bug", "First Bug", "Major", null, null, null, ADMIN_USERNAME, null, null, "1w", null, null);logWorkOnIssue(issueKey,"1d");gotoProjectBrowse(TEST_PROJECT_KEY);
Thursday, October 22, 2009
Atlassian Plugin SDK
$ atlas-unit-test$ atlas-clover$ atlas-integration-test--version--container--product
Thursday, October 22, 2009
Selenium
Thursday, October 22, 2009
Integration Testing Traps
• False Positives
• False Negatives
• Flapping
Thursday, October 22, 2009
Incomplete• Doesn’t test the full system
• JavaScript disabled, ignores browsers or app servers
• Passes when code is broken
• Solutions:
• HtmlUnit
• Selenium
• Cargo
• atlas-integration-test --container
Thursday, October 22, 2009
Entangled• Each test covers too much functionality
• Requires lots of setup to test each feature
• Bugs cause failures in tests for other features
• Solutions:
• Data loaders
• Remote APIs
• More specific test methods
• Domain-driven test frameworks
Thursday, October 22, 2009
Fragile
• Tests break due to UI changes
• Screen-scraping, complicated XPath
• Fails when not broken
• Solutions:
• Semantic markup (class and id attributes)
• Domain-driven test frameworks
Thursday, October 22, 2009
Timing or State Dependencies
• Inconsistent results from one test run to another
• Concurrency bugs, no clean up, environment issues
• Solutions:
• @Before/@After
• @BeforeClass/@AfterClass
• Continuous Integration
• Blood, sweat, and tears
Thursday, October 22, 2009
Slow
• Tests take too long to run
• Increases length of feedback loop
• Solutions:
• Clover test optimization
• Maven profiles
• Bamboo build trees
Thursday, October 22, 2009
Build TreesClover-Optimized
Unit Tests
All Unit TestsJava 5
IntegrationJava 5/Tomcat 6
IntegrationJava 5JBoss
All Unit TestsJava 6
IntegrationJava 5
Tomcat 5.5
IntegrationJava 6/Tomcat 6
IntegrationJava 6JBoss
IntegrationJava 6
Tomcat 5.5
Faster
Slower
Thursday, October 22, 2009
• False Positives
• False Negatives
• Flapping
←Test more
←Framework
←Get to work
Integration Testing Traps
Thursday, October 22, 2009
Resourceshttp://j.mp/plugin-testing
Thursday, October 22, 2009