testing grails: experiencies from the field
DESCRIPTION
Speaker: Colin Harrington Testing is built into grails, but many Grails apps go untested. We'll cover how to test many different artefacts as well cover many principles that have helped lead to succesfully tested Grails application.s Testing is built into grails, but many Grails apps go untested. We'll cover how to test many different artefacts as well cover many principles that have helped lead to succesfully tested Grails application. Adapted from a talk called "Testing the Crap out of your Grails application" We'll go beyond the basics and talk about my experiences in the 10+ Grails codebases and tems that I've had the opportunity to work with.TRANSCRIPT
Testing Grails
Source Code: github.com/zanthrash/loopback
Experiences from the Field
Colin Harrington :: SpringOne2gx 2011
Friday, October 28, 2011
Friday, October 28, 2011
Friday, October 28, 2011
Testing the CRAP out of your Grails App
Friday, October 28, 2011
whoamiColin Harrington
@ColinHarrington
[email protected]@objectpartners.com
Friday, October 28, 2011
“Testing is the engineering rigor of software development”
- Neal Ford
Friday, October 28, 2011
History
Friday, October 28, 2011
Pre Grails 1.0Testing Sucked
Friday, October 28, 2011
Friday, October 28, 2011
Grails 1.0
Friday, October 28, 2011
2.0Changes &
Enhancements
Friday, October 28, 2011
• Testing Basics
• Stubs, Spies, & Mocks
• Unit Testing
• Integration Testing
• Functional Testing
• What’s changing in Grails 2.0
• Grails Testing Ecosystem
• Experiences
Agenda
Friday, October 28, 2011
Friday, October 28, 2011
1st Class Citizen
• create-* scripts
• baked in the framework
• JUnit
• grails.test.*
Friday, October 28, 2011
Class Hierarchy
Friday, October 28, 2011
Running Tests All tests grails test-app
Failed test grails test-app -rerun
Specific test class grails test-app PresentationController
Specific test case grails test-app PresentationController.testFoo
Only unit test grails test-app unit:
Only integration test grails test-app integration:
Only Controller test grails test-app *Controller
Only test in a package grails test-app com.foo.*
Show output grails test-app -echoOut -echoErr
Friday, October 28, 2011
Phases & Types
Phases
unit
integration
functional
other
Types
unit
integration
functional
spock
custom
etc.
:http://ldaley.com/post/615966534/custom-grails-test
Friday, October 28, 2011
Unit Testing• No I/O
• Prove the basic correctness of the system
• State Testing
• Collaboration Testing
‣ Test Behavior NOT Implementation
Friday, October 28, 2011
void testAddSucceedsWhenSpeakerIsFound() {controller.params.title = "Test Presentation"def user = new User(id:1)
controller.springSecurityService = [currentUser: user] controller.accessCodeService = [createFrom: {title, event -> "abcd1234"}] mockDomain Presentation
controller.add()
controller.with{ assert redirectArgs.size() == 2 assert redirectArgs.controller == "speaker" assert redirectArgs.action == "index" assert flash.message == "'Test Presentation' added with access code: abcd1234" }}
Friday, October 28, 2011
Setup and Teardown• Always call super.setUp() and super.tearDown()
• super.setUp() creates:
• mock the applicationContext
• container for loaded codecs
• container for saved meta classes
• container for errorMap (for validation errors)
• Registers Sets, Lists, Maps, Errors so can be converted to XML and JSON
Friday, October 28, 2011
Setup and Teardown
• super.tearDown():
• removes the mocked meta-class from the GroovySystem.metaClassRegistry
• sets the original meta-class back in the GroovySystem.metaClassRegistry
• resets the mocked domain class id’s
Friday, October 28, 2011
Test Doubles
•Stubs
•Spies
•Mocks
Friday, October 28, 2011
Stubs
• A stand in for the real implementation with the simplest possible implementation
controller.accessCodeService = [ convertFrom: {title, event -> '1234abc'} ]
Friday, October 28, 2011
Spies• Stand in to capture an observation point
• Useful when:
• there is no output to inspect
• only concerned that an associated method or closure is called
Friday, October 28, 2011
Test Spy: Map with closure
void testControllerWithServiceCall() { def fooCalled = false controller.myService = [foo:{ a -> fooCalled = true}]
controller.methodThatCallsFooOnService()
assertTrue(fooCalled)}
Friday, October 28, 2011
Test Spy: mockFor(Class, loose=true)
void testControllerWithServiceCall() { def fooCalled = false
def mockService = mockFor(MyService, true) mockService.demand.foo() { a -> fooCalled = true} controller.myService = mockService.createMock()
controller.methodThatCallsFooOnService()
assertTrue fooCalled
}
Friday, October 28, 2011
Mocks
• mockFor()
• Used to contain the surprises & control:
• input params
• output
• method call count
• method call order
Friday, October 28, 2011
mockFor
• strict vs. loose
• createMock()
• verify()
• AVOID doing MetaClass programming yourself!
Friday, October 28, 2011
Strict Mocks: mockFor(Class)
void testControllerWithServiceCall() {
def mockService = mockFor(MyService) mockService.demand.bar(2..2) { a -> return a + 1} controller.myService = mockService.createMock()
controller.methodThatCallsBarOnService()
assert controller.redirectArgs.action == ‘foo’
mockService.verify()
}
Friday, October 28, 2011
Testing Domain Classes
• State Testing
• Constraint Testing
• mockForConstraintsTest()
• Domain Expectations Plugin
Friday, October 28, 2011
mockForConstraintsTest()
• Use in a GrailsUnitTestCase
• Think as double entry book keeping
• Signatures:
• mockForConstraintsTest( Domain )
• mockForConstraintsTest( Domain, [new Domain()]
✴ list param is used to test ‘unique’ constraint
Friday, October 28, 2011
mockForConstraintsTests(Class)
void testEventNameShouldNotValidateIfNull() { def event = new Event()
assertFalse event.validate() assertTrue event.hasErrors()
assert event.errors['name'] == 'nullable'}
Friday, October 28, 2011
Domain Expectations Plugin
• Designed to TDD your domain constraints
• Uses a GORM-like dynamic finder syntax
• expect[FieldName][Suffix]
• eg: user.expectAddressIsNotNullable
• Or a natural language like syntax
• eg: user.“address is not nullable”
Friday, October 28, 2011
Domain Expectations Example
void testEventNameConstraints() { Expectations.applyTo Event
def event = new Event()
event.expectNameIsNotNullable() event.expectNameIsNotBlank() event.expectNameHasMaxSize(40) event.expectNameHasAValidator()
event.expectNameIsValid(“jojo”)event.expectNameIsNotValid(“Bob”, ‘validator’)
}
Friday, October 28, 2011
Friday, October 28, 2011
Testing Controllers
• ControllerUnitTestCase (1.3.x)
• Mocking Domain Classes
• Mocking Services
• Mocking Grails Config
Friday, October 28, 2011
ContollerUnitTestCase
• Call super.setUp()
• Inspects the name of your test and instantiates an instance of your Controller Class and assigns it to the “controller” field.
• mocks out all the stuff in the controller class and adds helper methods for testing
Friday, October 28, 2011
What’s Mocked Out?Controller Methods
Helper MethodsHelper Methods
redirect getRedirectArgs getResponse
forward getForwardArgs getSession
chain getChainArgs getParams
render getRenderArgs getFlash
withFormat getTemplate getChainModel
withForm getModelAndView getActionName
log setModelAndView getControllerName
getRequest getServletContextFriday, October 28, 2011
Mocking Domain Objects
• mockDomain(Person)
• mockDomain(Person, [new Person()])
‣ used to simulate existing saved records
Friday, October 28, 2011
What you get
findAll load validate delete
findAllWhere getAll getErrors discard
findAllBy* ident hasErrors refresh
findBy* exists setErrors attach
get count clearErrors addTo*
read list save removeFrom*
create
= don’t do anything in testingFriday, October 28, 2011
• String based HQL queries
• Composite Identifiers
• Dirty checking methods
• Direct interaction with Hibernate
What you don’t get
Friday, October 28, 2011
Controller Unit Test Example
void testAddSucceedsWhenSpeakerIsFound() { controller.params.title = "Test Presentation" def user = new User(id:1) controller.springSecurityService = [currentUser: user] controller.accessCodeService = [createFrom: {title, event -> "abcd1234"}] mockDomain Presentation
controller.add()
controller.with{ assert redirectArgs.size() == 2 assert redirectArgs.controller == "speaker" assert redirectArgs.action == "index" assert flash.message == "'Test Presentation' added with access code: abcd1234" }}
Friday, October 28, 2011
Mocking Grails Config
mockConfig(''' braintree { merchantKey = "abcd1234" }''')
With String
With actual Config.groovy file *
mockConfig( new File('grails-app/conf/Config.groovy').text)
* violates No I/O ruleFriday, October 28, 2011
Testing URL Mappings
• GrailsUrlMappingsTestCase
• UrlMappingsUnitTestMixin (2.0+)
• Forward, Reverse, View and Response Code mappings
Friday, October 28, 2011
UrlMappings.groovyclass UrlMappings {! static mappings = {! ! "/$controller/$action?/$id?"{! ! ! constraints {! ! ! ! // apply constraints here! ! ! }! ! }! ! "/login"(controller: 'login', action:'auth')
! ! "/comment/$action/$accessCode"(controller: 'comment')
! ! "/"(controller:'speaker', action:"index")! ! "500"(view:'/error')! ! "404"(view:'/error404')! }}
Friday, October 28, 2011
Testing URLMappings
void testLoginPage() { assertUrlMapping(“/login”, controller:‘login’, action: ‘auth’)}
void testErrorPages() { assertUrlMapping(500, view: ‘error’) assertUrlMapping(404, view: ‘error404’)}
void testSiteRoot() { assertUrlMapping(‘/’, controller: ‘speaker’, action: ‘index’)}
Friday, October 28, 2011
Testing URLMappingsvoid testCommentMappings() {! assertUrlMapping("/comment/code/12345", controller: 'comment', action: 'code') {! ! accessCode = "12345"! }! assertUrlMapping("/comment/post/12345", controller: 'comment', action: 'post') {! ! accessCode = "12345"! }}
void testDefaultControllerActionIdMapping() {! assertUrlMapping("/event/show/3210", controller: 'event', action: 'show') {! ! id = 3210!!! }! assertUrlMapping("/presentation/show/125", controller: 'presentation', action: 'show') {! ! id = 125! }}
Friday, October 28, 2011
Testable Mappings
• Forward
• Reverse
• Status Code
• Views
Friday, October 28, 2011
static mappings = {
"/product/$id"(controller: "product") { action = [GET: "show", PUT: "update", DELETE: "delete", POST: "save"] }!}
No HTTP Methods support until 2.0
Friday, October 28, 2011
Friday, October 28, 2011
Testing Taglibs
• unit or integration
• TagLibUnitTestCase
• GroovyPagesTestCase
Friday, October 28, 2011
Unit Testing TagLibsclass LoopbackTagLibTests extends TagLibUnitTestCase {! protected void setUp() {! ! super.setUp()! }
! void testFixedWidthIpDefaultsTag() {! ! tagLib.fixedWidthIp(null, null)! ! assertEquals "................", tagLib.out.toString()! }
! void testFixedWidthIpParameters() {! ! tagLib.fixedWidthIp(ip: '127.0.0.1', width: '20', pad: '=', null)! ! assertEquals "127.0.0.1===========", tagLib.out.toString()
! ! shouldFail {! ! ! tagLib.fixedWidthIp(ip: '127.0.0.1', width: 'not-a-number', pad: '=', null)! ! }! }}
Friday, October 28, 2011
Integration Testing TagLibs
import grails.test.*
class LoopbackTagLibGSPTests extends GroovyPagesTestCase {
! void testFixedWidthIp() {! ! def template = '<g:fixedWidthIp ip="127.0.0.1" width="21" pad="-"/>'
! ! assertOutputEquals('127.0.0.1------------', template)! }
}
Friday, October 28, 2011
Testing GSPs
• Integration Tests
• GroovyPagesTestCase
• Views
• Templates
• assertOutputEquals(expected, template, model)
Friday, October 28, 2011
Integration Testing
• Domain Objects. Grails vs Rails
• Services
• Plugin utilization
• Rendering of Views
slower than unit testsFriday, October 28, 2011
Testing Services
• Autowired Beans
• Transactionality
• Full Database interactions
• Fixtures + Build Test Data
Friday, October 28, 2011
Functional Testing
• Testing from the outside
Fully functional app
Real database
Real HTTP traffic
Real integrations
Geb
Canoo Webtest
Selenium RC
EasyB
Fitness
BDD
Friday, October 28, 2011
Geb + Spockimport geb.spock.GebSpec
class MySpec extends GebSpec {
def "test something"() { when: go "/help"
then: $("h1").text() == "Help" }
}
Friday, October 28, 2011
Spock
• Highly expressive specification language
• Given ➡ When ➡ Then
• Drop in replacement for Testing Plugin for both Unit & Integration test
• Simple but detailed assertions
• Parameterized testing with wiki like syntax
Friday, October 28, 2011
Testing with Spock: Parallel Hierarchy
Testing Plugin Spock Plugin
GrailsUnitTestCase UnitSpec
ControllerUnitTestCase ControllerSpec
TagLibUnitTestCase TagLibSpec
GroovyTestCase IntegrationSpec
GroovyPagesTestCase GroovyPagesSpec
Friday, October 28, 2011
Spock Parameterized Test Example: class ParameterizedSpec extends UnitSpec{
@Unroll("#a + #b = #c") def "summation of 2 numbers"() { expect: a + b == c
where: a |b |c 1 |1 |2 2 |1 |3 3 |2 |6 }}
Condition not satisfied:a + b == c| | | | |3 5 2 | 6 false
Friday, October 28, 2011
Spock ControllerSpec Example: class PresentationControllerSpec extends ControllerSpec{
@Shared def user = new User(id:1)
@Unroll("add presentation for user: #testUser with title: #title") def "call to add a new presentation for logged in user"() { given:"collaborators are mocked" mockDomain Presentation mockDomain Speaker, [new Speaker(user:user)] controller.springSecurityService = [currentUser: testUser ] controller.accessCodeService = [createFrom:{title, event -> "1234abcd"}] and:"prarams are set" controller.params.event = "Event Name" controller.params.date = new Date() controller.params.title = title when:"call the action" controller.add()
then:"should be redirected to '/speaker/index'" controller.redirectArgs.controller == 'speaker' controller.redirectArgs.action == 'index'
and: "should have correct flash message" controller.flash.message == expectedFlashMessage
where: title | testUser |expectedFlashMessage 'test' | user |"'test' added with access code: 1234abcd" null | user |"Could not add null to your list at this time" 'test' | null |"Could not find a Speaker to add test to your list at this time" }}
Friday, October 28, 2011
2.0 Changes
• Testing Hierarchy is dead
• Replaced with annotated mix-ins
• In-memory GORM implementation
• Unit Tests finally use GrailsMockHttpServletReponse
Friday, October 28, 2011
2.0 Testing Annotations & Mixins
@TestFor(ClassUnderTest)
@Mock(Class) OR @Mock([Class1, Class2])
@TestMixin(Mixin)
- DomainClassUnitTestMixin- ControllerUnitTestMixin- FiltersUnitTestMixin- GroovyPageUnitTestMixin- ServiceUnitTestMixin- UrlMappingsUnitTestMixin
Friday, October 28, 2011
2.0 Domain Constraints Example
@TestFor(Event)class EventTests {
@Before void setup() { mockForConstraintsTests Event }
@Test void eventNameShouldNotValidateIfNull() { def event = new Event()
assertFalse event.validate() assertTrue event.hasErrors() assert event.errors['name'] == 'nullable' }}
Friday, October 28, 2011
2.0 Controller Example@TestFor(PresentationController)@Mock([User, Speaker, Presentation])class PresentationControllerTests {
User user
@Before void setUp() { controller.params.title = "Test Presentation" user = new User(id:1) new Speaker(user:user).save(validate: false) }
@Test void testAddSucceedsWhenSpeakerIsFound() { controller.springSecurityService = [currentUser: user] controller.accessCodeService = [createFrom: {title, event -> "abcd1234"}]
controller.add()
assert response.redirectedUrl == '/speaker/index' assert flash.message == "'Test Presentation' added with access code: abcd1234" }}
Friday, October 28, 2011
Friday, October 28, 2011
Ingredients for Success
• Jenkins
• Git / VCS
• Servers
Friday, October 28, 2011
Tools
Friday, October 28, 2011
Plugins
• Spock (http://code.google.com/p/spock/)
• Spock Plugin (http://www.grails.org/plugin/spock)
• Geb (http://geb.codehaus.org/latest/index.html)
• Domain Expectation Plugin (http://www.grails.org/plugin/domain-expectations)
• Build Test Data Plugin (http://www.grails.org/plugin/build-test-data)
• Remote Control Plugin (http://www.grails.org/plugin/remote-control)
• Fixtures Plugin (http://www.grails.org/plugin/fixtures)
• Greenmail Plugin (http://www.grails.org/plugin/greenmail)
Friday, October 28, 2011
Perspective
Friday, October 28, 2011
Spectrum
• Run-before-checkin
• Continuous Testing (on each checkin)
• Scheduled Testing
• Load Testing?
• Manual Testing
• Regression Testing
Friday, October 28, 2011
The Future is now
• Git + Gerrit + Jenkins
http://alblue.bandlem.com/2011/02/gerrit-git-review-with-jenkins-ci.html
http://www.infoq.com/articles/Gerrit-jenkins-hudson
Friday, October 28, 2011
Considerations:• Passing Tests != stable != quality // Think!
• Test your Migrations { structure, data }
• Finite Resources
• Developers fight with ‘Done’
• Lost in the weeds.
• lack of skill with the tools, (its OK)
• Professional Yak shaving
Friday, October 28, 2011
Code Coverage• Focus on test quality NOT test coverage
void testWhyCodeCoverageNumbersAreCrap() { def service = new AccessCodeService() service.metaClass.methods.*name.sort().unique().each { try{ service.invokeMethod(it, ['blah', 'blah']) } catch (Exception ex ){ //eat it } } }
Friday, October 28, 2011
Now what?
• Download the Testing Cheat Sheet
• http://bit.ly/GRAILSTESTING
• Look into Geb and Spock
• Take the small steps forward!
Source Code: github.com/zanthrash/loopbackFriday, October 28, 2011
Questions ?
Thanks
Friday, October 28, 2011
Friday, October 28, 2011
Thank you
Friday, October 28, 2011