groovy testing aug2009

2006-2

Dr Paul King

paulk@asert.com.au

@paulk_asert

ASERT, Australia

How to make your Testing

more GroovyCraig Smith

craig.smith@suncorp.com.au

@smithcdau

Suncorp, Australia

Topics• Why Groovy for Testing?

• Groovy Intro

• Web Drivers

• Test Runners

• Non-web Drivers

• Other Tools

• Going beyond

Agile2009 - 2

2006-2

Focus is on using the Groovy dynamic language

for primarily functional and acceptance testing

with a forward looking perspective. Also considers

polyglot options. The techniques and lessons

learned can be applied to other kinds of testing

and are also applicable to similar languages.

Topics not covered in detail

• Coverage– But coverage options available

• Mocking– But built-in support and many libraries available

• CI Support– Support in Hudson, Team City, Anthill Pro to call

Groovy

• Code Metrics– CodeNarc and other tools available

• Build Tools– Ant, Maven, GMaven, Gant, Gradle, ...

Agile2009 - 3

2006-2

TopicsWhy Groovy for Testing?

• Groovy Intro

• Web Drivers– Native Groovy, HttpBuilder, HtmlUnit

WebTest, Watij, Selenium, WebDriver

Tellurium, JWebUnit

• Test Runners– Native Groovy, JUnit, TestNG, Spock, EasyB, JBehave, Cucumber

• Non-web Drivers– SOAP/REST, Database, FEST, ...

• Other Tools– SoapUI, ITest2, JMeter, Twist, ...

• Going beyond– Polyglot, Model-driven, Constraint/logic languages, Rules

Agile2009 - 4

2006-2

Why Test With Groovy?

?Agile2009 - 5

2006-2

• Wrong first question!

• Perfect second question?

XConsider first the task at hand and the

organization and people fit!Agile2009 - 6

2006-2

“People Fit” / “Organization Fit”

• People– Developers (familiarity with languages)

– Testers (tools language familiarity)

– Responsibility to create/run/maintain

– BAs/SMEs (ability to read technical tests)

– Consider both development and maintenance

– Expected feedback from tests (JUnit/Reports)

• Organization– Maturity level

– Degree of automation

– Tools in use

– Need for adaptability

Agile2009 - 7

2006-2

• Advantages– Easy to learn

– Particularly good fit if the application you are testing

is built for the JVM or your developers work with Java

/ JVM languages

– Supports polyglot programming when needed

– Easy to plug and play different testing tools

– Good community & tools for professional agile

• Disadvantages– Requires JVM

– Less advantages if your developers using Python,

.Net, PHP

– Maybe your testers already know Ruby

Agile2009 - 8

2006-2

Scripting/Dynamic Languages

• Advantages– Lend themselves to succinct code/DSLs

– Powerful

– Increased Refactoring

– Increased Reuse

– Less prone to Brittleness

– Flexibility for tool integration

– Open APIs provide extensibility

• Disadvantages– Can be too low level (but many options now)

– Sometimes less tool support (but changing now)

Agile2009 - 9

2006-2

Test Characteristics• Coverage/Traceability

– Requirement coverage/traceability

– Code coverage: line, branch, path, state

– Transactional Tracing

• Purpose– Unit, Integration, System, Customer

• Manageability– Removing duplication

– Managing lifecycle: shared setUp/tearDown (@Before @After)

• Handling test data– Data-driven, databases, Spreadsheets, tables, keywords, random, model-driven

• Large Test Volumes– Speed of feedback, performance testing

• Tool evolution/longevity/cost/support/documentation– Open Source/Commercial, Critical mass/popularity

• Combinability

Agile2009 - 10

2006-2

Key Testing Practices • Use testing DSL’s

• Look to move up the testing stack– It used to be all about the driver

– Now the driver is hidden in the framework or tool stack

• Apply good testing practices– Pareto analysis, bug clusters, mutation testing, test early

– All pairs/equivalence partitions/orthogonal array testing

– Risk-based test selection, coding for testability, use CI

– Boundary value analysis, defensive programming

• Plug and play testing tools– Run different drivers with different runners and different tools

• Complement automated tests with exploration

• Expand testing scope– Test environment readiness, test deployments

Agile2009 - 11

2006-2

Groovy and Testing Tool Spectrum*

Agile2009 - 12

2006-2

Database

Drivers

DbUnit

DataSets

SqlUnit

groovy.sql

BigTable

SOAP /

Drivers

GroovyWS

XML-RPC

JAX-WS

JAX-RS

Utilities

All Pairs Combinations

Polyglot languages

Logic programming

Threads, Parallel /

Concurrency libraries

Data-driven libraries

Networking libraries

XML Processing

Read/write files /

Excel / Word / CSV

Reporting, Logging

Drivers

AntUnit

Telnet

ExecTools

iTest2, SoapUI, Twist,

IDEs, JMeter, Text

editors, Recorders,

Build Tools, CI

Drivers

WebTest

WebDriver

JWebUnit

Tellurium

Selenium

HtmlUnit

HttpBuilder

Cyberneko

Runners

Native Groovy, JUnit, TestNG, Spock, EasyB,

JBehave, Cucumber, Robot Framework

* Tools/libraries/frameworks don't always neatly fall into one category – still useful conceptually

Groovy Intro

Tellurium, JWebUnit

Agile2009 - 13

2006-2

Agile2009 - 14

2006-2

What is Groovy?

• “Groovy is like a super version

of Java. It can leverage Java's

enterprise capabilities but also

has cool productivity features like closures,

DSL support, builders and dynamic typing.”

Groovy = Java – boiler plate code+ optional dynamic typing+ closures+ domain specific languages+ builders+ metaprogramming

Agile2009 - 15

2006-2

Groovy Goodies Overview• Fully object oriented

• Closures: reusable

and assignable

pieces of code

• Operators can be

overloaded

• Multimethods

• Literal declaration for

lists (arrays), maps,

ranges and regular

expressions

• GPath: efficient

object navigation

• GroovyBeans

• grep and switch

• Templates, builder,

swing, Ant, markup,

XML, SQL, XML-RPC,

Scriptom, Grails,

tests, Mocks

Growing Acceptance …

A slow and steady start but now gaining in

momentum, maturity and mindshare

Making

Groovy

(soon)

Now free

… Growing Acceptance …

2006-2

Agile2009 - 17

2006-2

Agile2009 - 18

Groovy and

Grails downloads:

70-90K per month

and growing

2006-2

Agile2009 - 19

Source: http://www.grailspodcast.com/

Source: http://www.micropoll.com/akira/mpresult/501697-116746

2006-2

Agile2009 - 20

http://www.java.net

http://www.jroller.com/scolebourne/entry/devoxx_2008_whiteboard_votes

2006-2

Agile2009 - 21http://www.leonardoborges.com/writings

What alternative JVM language are you using or intending to use

2006-2

Agile2009 - 22

http://it-republik.de/jaxenter/quickvote/results/1/poll/44 (translated using http://babelfish.yahoo.com)

… Growing Acceptance

Agile2009 - 23

2006-2

Agile2009 - 24

2006-2

The Landscape of JVM Languages

Java bytecode calls

for static types

Dynamic features call

for dynamic types

optional

static

The terms “Java Virtual Machine” and “JVM” mean a Virtual Machine for the Java™ platform.

Agile2009 - 25

2006-2

Groovy StarterSystem.out.println("Hello, World!"); // optional semicolon,println 'Hello, World!' // System.out, brackets,

// main() method, class defn

def name = 'Guillaume' // dynamic typingprintln "$name, I'll get the car." // GString

String longer = """${name}, the caris in the next row.""" // multi-line string

// with static typing

assert 0.5 == 1/2 // BigDecimal equals()

def printSize(obj) { // optional duck typingprint obj?.size() // safe dereferencing

def pets = ['ant', 'bee', 'cat'] // native list syntaxpets.each { pet -> // closure support

assert pet < 'dog' // overloading '<' on String} // or: for (pet in pets)...

A Better Java...

Agile2009 - 26

2006-2

import java.util.List;import java.util.ArrayList;

class Erase {private List removeLongerThan(List strings, int length) {

List result = new ArrayList();for (int i = 0; i < strings.size(); i++) {

String s = (String) strings.get(i);if (s.length() <= length) {

result.add(s);}

}return result;

}public static void main(String[] args) {

List names = new ArrayList();names.add("Ted"); names.add("Fred");names.add("Jed"); names.add("Ned");System.out.println(names);Erase e = new Erase();List shortNames = e.removeLongerThan(names, 3);System.out.println(shortNames.size());for (int i = 0; i < shortNames.size(); i++) {

String s = (String) shortNames.get(i);System.out.println(s);

This code

is valid

Java and

valid Groovy

Based on an

example by

Jim Weirich

& Ted Leung

...A Better Java...

Agile2009 - 27

2006-2

import java.util.List;import java.util.ArrayList;

List result = new ArrayList();for (int i = 0; i < strings.size(); i++) {

String s = (String) strings.get(i);if (s.length() <= length) {

result.add(s);}

}return result;

}public static void main(String[] args) {

List names = new ArrayList();names.add("Ted"); names.add("Fred");names.add("Jed"); names.add("Ned");System.out.println(names);Erase e = new Erase();List shortNames = e.removeLongerThan(names, 3);System.out.println(shortNames.size());for (int i = 0; i < shortNames.size(); i++) {

String s = (String) shortNames.get(i);System.out.println(s);

Do the

semicolons

add anything?

And shouldn‟t

we us more

modern list

notation?

Why not

import common

libraries?

...A Better Java...

Agile2009 - 28

2006-2

List result = new ArrayList()for (String s in strings) {

if (s.length() <= length) {result.add(s)

}}return result

public static void main(String[] args) {List names = new ArrayList()names.add("Ted"); names.add("Fred")names.add("Jed"); names.add("Ned")System.out.println(names)Erase e = new Erase()List shortNames = e.removeLongerThan(names, 3)System.out.println(shortNames.size())for (String s in shortNames) {

System.out.println(s)}

...A Better Java...

Agile2009 - 29

2006-2

List result = new ArrayList()for (String s in strings) {

if (s.length() <= length) {result.add(s)

}}return result

public static void main(String[] args) {List names = new ArrayList()names.add("Ted"); names.add("Fred")names.add("Jed"); names.add("Ned")System.out.println(names)Erase e = new Erase()List shortNames = e.removeLongerThan(names, 3)System.out.println(shortNames.size())for (String s in shortNames) {

Do we need

the static types?

Must we always

have a main

method and

class definition?

How about

improved

consistency?

...A Better Java...

Agile2009 - 30

2006-2

def removeLongerThan(strings, length) {def result = new ArrayList()for (s in strings) {

if (s.size() <= length) {result.add(s)

}}return result

names = new ArrayList()names.add("Ted")names.add("Fred")names.add("Jed")names.add("Ned")System.out.println(names)shortNames = removeLongerThan(names, 3)System.out.println(shortNames.size())for (s in shortNames) {

...A Better Java...

Agile2009 - 31

2006-2

def removeLongerThan(strings, length) {def result = new ArrayList()for (s in strings) {

if (s.size() <= length) {result.add(s)

}}return result

names = new ArrayList()names.add("Ted")names.add("Fred")names.add("Jed")names.add("Ned")System.out.println(names)shortNames = removeLongerThan(names, 3)System.out.println(shortNames.size())for (s in shortNames) {

Shouldn‟t we

have special

notation for lists?

And special

facilities for

list processing?

Is „return‟

needed at end?

...A Better Java...

Agile2009 - 32

2006-2

def removeLongerThan(strings, length) {strings.findAll{ it.size() <= length }

names = ["Ted", "Fred", "Jed", "Ned"]System.out.println(names)shortNames = removeLongerThan(names, 3)System.out.println(shortNames.size())shortNames.each{ System.out.println(s) }

...A Better Java...

Agile2009 - 33

2006-2

def removeLongerThan(strings, length) {strings.findAll{ it.size() <= length }

names = ["Ted", "Fred", "Jed", "Ned"]System.out.println(names)shortNames = removeLongerThan(names, 3)System.out.println(shortNames.size())shortNames.each{ System.out.println(s) }

Is the method

now needed?

Easier ways to

use common

methods?

Are brackets

required here?

...A Better Java...

Agile2009 - 34

2006-2

names = ["Ted", "Fred", "Jed", "Ned"]println namesshortNames = names.findAll{ it.size() <= 3 }println shortNames.size()shortNames.each{ println it }

...A Better Java

Agile2009 - 35

2006-2

names = ["Ted", "Fred", "Jed", "Ned"]println namesshortNames = names.findAll{ it.size() <= 3 }println shortNames.size()shortNames.each{ println it }

["Ted", "Fred", "Jed", "Ned"]3TedJedNed

Better Lists, Maps, Ranges

• Lists– Special syntax for list literals

– Additional common methods (operator overloading)

• Maps– Special syntax for map literals

– Additional common methods

• Ranges– Special syntax for various kinds of ranges

Agile2009 - 36

2006-2

def list = [3, new Date(), 'Jan']assert list + list == list * 2

def map = [a: 1, b: 2]assert map['a'] == 1 && map.b == 2

def letters = 'a'..'z'def numbers = 0..<10

Closures

• Traditional mainstream languages– Data can be stored in variables, passed around,

combined in structured ways to form more complex

data; code stays put where it is defined

• Languages supporting closures– Data and code can be stored in variables, passed

around, combined in structured ways to form more

complex algorithms and data

Agile2009 - 37

2006-2

doubleNum = { num -> num * 2 }println doubleNum(3) // => 6

processThenPrint = { num, closure ->num = closure(num); println "num is $num"

}processThenPrint(3, doubleNum) // => num is 6processThenPrint(10) { it / 2 } // => num is 5

SwingXBuilder

Agile2009 - 38

2006-2

import groovy.swing.SwingXBuilderimport static java.awt.Color.*import static java.lang.Math.*

def swing = new SwingXBuilder()def frame = swing.frame(size: [300, 300]) {

graph(plots: [[GREEN, {value -> sin(value)}],[BLUE, {value -> cos(value)}],[RED, {value -> tan(value)}]

])}.show()

• Groovy Intro

Web Drivers– Native Groovy, HttpBuilder, HtmlUnit

Tellurium, JWebUnit

Agile2009 - 39

2006-2

Agile2009 - 40

2006-2

Concept

Driver

Runner<webtest name="myTest">

<steps>

<invoke

description="get Login Page"

url="login" />

<verifyTitle

description="we should see the login title"

text="Login Page" />

</steps>

</webtest>

Web Server

HTTP Request / Response

Script

Manual

Automated

Driver Category• Real browser invoker

– Runs on platform

supported by real

browser

– May need multiple

platforms, e.g. IE6/IE7

– Uses actual JavaScript

engine

– Can be easier to use

with test recorders

– Automation

capabilities differ

across browsers

– Can typically get to all

aspects of browser

• Browser Emulators

– Can simulate multiple

browsers

– Less platform

restrictions

– Good for CI

– Easier to not download

images, resources

– Ability to optimise

JavaScript interactions

– More extensible

– Ability to disable

JavaScript

– Scope for parallelism

Agile2009 - 41

2006-2

Agile2009 - 42

2006-2

Application under Test…

Agile2009 - 43

2006-2

…Application under Test...

Agile2009 - 44

2006-2

…Application under Test

Native Groovy...

• Access URLs

• Built-in XML parsing

• Built-in friendly regular expression syntax

• Even for advanced cases, there is friendly

access to low-level things:– Sockets, Processes

– Databases and other things

– Files

• Huge range of Java libraries– PDF

– Reading, writing Excel

Agile2009 - 45

2006-2

...Native Groovy...

• Useful URL methods

Agile2009 - 46

2006-2

def html = new URL('http://localhost:8080').text

assert html.contains('<title>Welcome to SimpBlog</title>')

html.find(~'<title>(.*)</title>') { all, title ->assert title == 'Welcome to SimpBlog'

...Native Groovy...

• Built-in XML Parsing

Agile2009 - 47

2006-2

def page = newXmlSlurper().parse('http://localhost:8080/viewPost?id=1')

assert page.body.h1.text().contains('Tis the season')assert page.body.h3[1].text() == 'Category: Home'assert page.body.h3[2].text() == 'Author: Bart'assert page.body.table.tr.td.p.text() ==

"Aren't we forgeting the true meaning of Christmas? You know, the birth of Santa."

...Native Groovy

• Easy access to Java libraries

Agile2009 - 48

2006-2

@Grab('nekohtml:nekohtml:1.9.6.2')import org.cyberneko.html.parsers.SAXParser

def parser = new XmlSlurper(new SAXParser())def page = parser.parse('http://localhost:8080/viewPost?id=1')assert page.BODY.H1.text().contains('Tis the season')assert page.BODY.H3[1].text() == 'Category: Home'assert page.BODY.H3[2].text() == 'Author: Bart'assert page.BODY.TABLE.TR.TD.P.text() ==

"Aren't we forgeting the true meaning of Christmas? You know, the birth of Santa."

• Builder for Http interactions– Flexible: bogus posts, response codes, JSON, non-

HttpBuilder

2006-2

@Grab(group='org.codehaus.groovy.modules.http-builder',module='http-builder', version='0.5.0-RC1')

import groovyx.net.http.*import static groovyx.net.http.ContentType.URLENC

def http = new HTTPBuilder('http://localhost:8080')def postBody = [title:'Bart was here (and so was HttpBuilder)',

content:'Cowabunga Dude!', author:'1', category:'3']http.post(path:'/addPost', body: postBody,

requestContentType: URLENC) { resp, html ->assert resp.contentType == 'text/html'assert resp.status == 200assert html.BODY.H1.text().matches('Post.*: Bart was here.*')assert html.BODY.H3[1].text() == 'Category: Home'assert html.BODY.H3[2].text() == 'Author: Bart'assert html.BODY.TABLE.TR.TD.P.text() == 'Cowabunga Dude!'

}Agile2009 - 49

HtmlUnit• 100% Java-based headless browser emulator

– Can test any Web site: Java, .Net, PHP, Rails, ...

• Open Source– Apache 2 license

– Hosted at SourceForge

– 7 committers (3 very active)

– Very mature

• Useful for:– Integration and acceptance testing

– Screen scraping, deployment automation, ...

• Used by other drivers:– Canoo WebTest , JWebUnit , WebDriver , JSFUnit , Celerity

• Special features:– Easy ajax mode, emulation of multiple browsers

Agile2009 - 50

2006-2

HtmlUnit Features...• Support for the HTTP and HTTPS protocols

• Support for cookies

• Ability to specify whether failing responses from

the server should throw exceptions or should be

returned as "error" pages

• Support for submit methods POST and GET– As well as HEAD, DELETE, ...

• Ability to customize the request headers being

sent to the server

• Support for HTML responses– Wrapper for HTML pages that provides easy access to all

information contained inside them

– Support for submitting forms and clicking links

– Support for walking the DOM model of HTML documentsAgile2009 - 51

2006-2

...HtmlUnit Features• Proxy server support

• Support for basic and NTLM authentication

• Excellent JavaScript support– jQuery 1.2.6: Full support

– MochiKit 1.4.1: Full support

– GWT 1.6.4: Full support

– Sarissa 0.9.9.3: Full support

– MooTools 1.2.1: Full support

– Prototype 1.6.0: Very good support

– Ext JS 2.2: Very good support

– Dojo 1.0.2: Good support

– YUI 2.3.0: Good support

Agile2009 - 52

2006-2

HtmlUnit: Testing New Blog Post...

Agile2009 - 53

2006-2

@Grab('net.sourceforge.htmlunit:htmlunit:2.5')import com.gargoylesoftware.htmlunit.WebClient

def client = new WebClient()def page = client.getPage('http://localhost:8080/postForm')// check page titleassert 'Welcome to SimpBlog' == page.titleText

// fill in blog entry and post itdef form = page.getFormByName('post')form.getInputByName('title').

setValueAttribute('Bart was here (and so was HtmlUnit)')form.getSelectByName('category').getOptions().find{

it.text == 'Home' }.setSelected(true)form.getTextAreaByName('content').setText('Cowabunga Dude!')def result = form.getInputByName('btnPost').click()

...HtmlUnit: Testing New Blog Post

Agile2009 - 54

2006-2

// check blog post detailsassert result.getElementsByTagName('h1').item(0).

textContent.matches('Post.*: Bart was here.*')def h3headings = result.getElementsByTagName('h3')assert h3headings.item(1).textContent == 'Category: Home'assert h3headings.item(2).textContent == 'Author: Bart'

// expecting:// <table><tr><td><p>Cowabunga Dude!</p></td></tr></table>def cell = result.getByXPath('//TABLE//TR/TD')[0]def para = cell.getFirstChild()assert para.textContent == 'Cowabunga Dude!'

Agile2009 - 55

2006-2

Canoo WebTest• Description– Open source tool for automated testing of web applications

– Declarative approach in XML or testing DSL in Groovy

– Has Test Recorder

– Excellent reporting options

– Ant-based under the covers

&config;

<steps>

<invoke stepid="get Login Page"

url="login.jsp" />

<verifytitle stepid="we should see the login title"

text="Login Page" />

<setinputfield stepid="set user name"

name="username"

value="scott" />

<setinputfield stepid="set password"

name="password"

value="tiger" />

<clickbutton stepid="Click the submit button"

label="let me in" />

<verifytitle stepid="Home Page follows if login ok"

text="Home Page" />

</steps>

</testSpec>

</target>

Canoo WebTest Features• Strongly encourages declarative testing

– Supports testing DSLs, test structuring and reuse through

macrodefs and imports for XML flavor &

methods and closures for Groovy flavor

• Extensive support for HTML pages– Including JavaScript

• Also supports other MIME types– Generically as binary streams

– Special support for PDF, Excel, Emails

• Ant heritage provides easy IDE/CI hooks

• Excellent Documentation

• Excellent Community

• Eats own dog food– high quality codebase

Agile2009 - 56

2006-2

Canoo WebTest Steps...

Agile2009 - 57

2006-2

...Canoo WebTest Steps...

Agile2009 - 58

2006-2

...Canoo WebTest Steps...

Agile2009 - 59

2006-2

...Canoo WebTest Steps

Agile2009 - 60

2006-2

WebTest: Testing New Blog Post...

Agile2009 - 61

2006-2

<setInputField name="title"

value="Bart was here (and so was WebTest)"/>

</group>

...WebTest: Testing New Blog Post...

Agile2009 - 62

2006-2

<verifyElementText type="h1" regex="true"

text="Post.*: Bart was here.*"/>

</group>

println "Test run at: ${new Date()}"

</groovy>

</webtest>

Agile2009 - 63

2006-2

ant.webtest(name: 'Test SimpBlog') {invoke url: "http://localhost:8080/",

description: "Home Page"verifyTitle text: "Welcome to SimpBlog"group description: "Post New Blog Entry", {clickLink label: "New Blog Entry"setInputField name: "title",

value: "Bart was here (and so was WebTest with Groovy)"setSelectField name: "category", text: "School"setInputField name: "content", value: "Cowabunga Dude!"clickButton name: "btnPost"

}group description: "Check Blog Post", {verifyElementText type: "h1", regex: "true",

text: "Post.*: Bart was here.*"verifyXPath xpath: "//h3[2]/text()", text: "Category: School"verifyXPath xpath: "//h3[3]/text()", text: "Author: Bart"verifyElementText type: "p", text: "Cowabunga Dude!"

Agile2009 - 64

2006-2

...WebTest: Testing New Blog Post

Agile2009 - 65

2006-2

Agile2009 - 66

2006-2

Firefox Recorder

• Description– Java API that provides control and automation of

Internet Explorer

– Supports actions like navigating, clicking links, filling

out forms, etc.

– Also supports more complex actions like file

downloading and uploading, popup windows and

dialogs, and screen captures

• Special Features– Ability to work with IE interactively

– Can attach to an existing browser session

– Special browser commands, e.g. ie.fullScreen(true)

– Handles child browsers and popup dialogs

Agile2009 - 67

2006-2

Watij Features

• Finders– tag(String tagName)

– attribute(String name, String value)

– index(int index)

– text(String text)

– name(String value)

– value(String value)

– caption(String value)

– id(String value)

– title(String value)

– alt(String value)

– src(String value)

– action(String value)

– method(String value)

– url(String value)

– href(String value)

– xpath(String expression)Agile2009 - 68

2006-2

Watij: Testing New Blog Post...

Agile2009 - 69

2006-2

import watij.runtime.ie.IEimport static watij.finders.SymbolFactory.*import static watij.finders.FinderFactory.*

def ie = new IE()ie.start('http://localhost:8080/postForm')

// check page titleassert ie.title() == 'Welcome to SimpBlog'

// fill in query form and submit itie.textField(name, 'title').

set('Bart was here (and so was Watij)')ie.textField(name, 'content').set('Cowabunga dude!')ie.selectList(name, "category").

option(text, "Home").select()ie.button(name, 'btnPost').click()

...Watij: Testing New Blog Post...

Agile2009 - 70

2006-2

// check entered post is being displayedassert ie.htmlElement(tag, 'H1').text().

matches('Post.*: Bart was here.*')def h3headers = ie.htmlElements(tag, 'H3')assert h3headers.get(1).text() == 'Category: Home'assert h3headers.get(2).text() == 'Author: Bart'

// try a more advanced finder// content is at: //TABLE/TBODY/TR/TD/Pdef row = ie.htmlElement(xpath('//TABLE/TBODY/TR'))assert row.cell(0).htmlElement(tag, 'P').text() ==

'Cowabunga dude!'

ie.close()

...Watij: Testing New Blog Post

Agile2009 - 71

2006-2

Selenium...

• Description– Tools to help

automate testing

for web-based

applications

– Support for

running tests on

multiple browser

platforms

• Components– Selenium Core

– Selenium IDE

Selenium RC

– Selenium Grid

Agile2009 - 72

2006-2

Source: http://seleniumhq.org/projects/remote-control/

Our focus

...Selenium

Agile2009 - 73

2006-2

Source: http://seleniumhq.org/docs/01_introducing_selenium.html

Selenium: Testing New Blog Post...

Agile2009 - 74

2006-2

import com.thoughtworks.selenium.DefaultSeleniumimport org.openqa.selenium.server.SeleniumServer

// start auxiliary serverdef server = new SeleniumServer()server.start()

// uncomment one of below//def browser = "*iexplore"//def browser = "*firefox3" // if Firefox already in your path//def browser = "*firefox3 C:/Program Files/Mozilla Firefox/firefox.exe"def browser = "*firefox3 C:/Program Files (x86)/Mozilla Firefox/firefox.exe"

def selenium = new DefaultSelenium("localhost", 4444, browser,"http://localhost:8080")

selenium.start()

...Selenium: Testing New Blog Post

Agile2009 - 75

2006-2

// post blogselenium.open "/postForm"selenium.type "title", "Bart was here (and so was Selenium)"selenium.select "category", "Home"selenium.type "content", "Cowabunga Dude!"selenium.click "btnPost"selenium.waitForPageToLoad "5000"

// checksassert selenium.isTextPresent('regex:Post.*: Bart was here')assert selenium.isElementPresent('//h3[text()="Author: Bart"]')assert selenium.isElementPresent('//h3[text()="Category: Home"]')assert selenium.isElementPresent(

'//table//tr/td/p[text()="Cowabunga Dude!"]')

selenium.stop()server.stop()

Selenium IDE

Agile2009 - 76

2006-2

Features:

• Easy record and playback

• Intelligent field selection will use

IDs, names, or XPath as needed

• Autocomplete for all common

Selenium commands

• Walk through tests

• Debug and set breakpoints

• Save tests as HTML, Ruby

scripts, or any other format

• Support for Selenium user-

extensions.js file

• Option to automatically assert the

title of every page

Selenium Other Tools...

Agile2009 - 77

2006-2

...Selenium Other Tools...

Agile2009 - 78

2006-2

...Selenium Other Tools

Agile2009 - 79

2006-2

Source: http://selenium-grid.seleniumhq.org/how_it_works.html

WebDriver

• Description– Simple API to drive both real browsers

• for testing javascript heavy apps

– and a pure 'in memory' emulator solution

• for faster testing of simpler apps

• uses HtmlUnit under the covers in emulator mode

– Represents next generation of Selenium RC

• though merging into Selenium may happen under the covers

if you are a Selenium user

– Roadmap has plans to leverage some advanced

Selenium like features

• RemoteWebDriver

• FarmedWebDriver (think Selenium Grid)

Agile2009 - 80

2006-2

WebDriver: Testing New Blog Post...

Agile2009 - 81

2006-2

import org.openqa.selenium.Byimport org.openqa.selenium.htmlunit.HtmlUnitDriver

def driver = new HtmlUnitDriver()driver.get('http://localhost:8080/postForm')assert driver.title == 'Welcome to SimpBlog'

// fill in query form and submit itdriver.findElement(By.name('title')).

sendKeys('Bart was here (and so was WebDriver)')driver.findElement(By.name('content')).

sendKeys('Cowabunga dude!')def select = driver.findElement(By.name('category'))select.findElements(By.tagName("option")).find{

it.text == 'Home' }.setSelected()driver.findElement(By.name('btnPost')).click()

...WebDriver: Testing New Blog Post

Agile2009 - 82

2006-2

assert driver.findElement(By.tagName("h1")).text.matches('Post.*: Bart was here.*')

def h3headers = driver.findElements(By.tagName("h3"))assert h3headers[1].text == 'Category: Home'assert h3headers[2].text == 'Author: Bart'

// try a more advanced finder// content is at: //TABLE/TBODY/TR/TD/Pdef row = driver.findElement(By.xpath("//table/tbody/tr"))def col = row.findElement(By.tagName("td"))def para = col.findElement(By.tagName("p"))assert para.text == 'Cowabunga dude!'

Tellurium

• Description– built on top of Selenium but tries to solve several

shortcomings

– "record and reply" style, difficult to refactor and

maintain, so instead define UI components

declaratively then write tests in terms of UI

– Provides many predefined UI objects for you to use

directly, such as Button, CheckBox, InputBox,

Selector, TextBox, and Table but also ability to write

your own custom UI objects

– Supports advanced locating mechanisms: composite

locator, "group locating"

– Supports testing DSL

– Supports data-driven tests

Agile2009 - 83

2006-2

Architecture

Agile2009 - 84

2006-2

Source: http://code.google.com/p/aost/wiki/Introduction

Tellurium Example

• Selenium:

• Tellurium UI:

• Tellurium DSL Test:

Agile2009 - 85

2006-2

selenium.type("//input[@title='Google Search']", input)selenium.click("//input[@name='btnG' and @type='submit']")

ui.Container(uid: "google_start_page",clocator: [tag: "td"], group: "true") {

InputBox(uid: "searchbox",clocator: [title: "Google Search"])

SubmitButton(uid: "googlesearch",clocator: [name: "btnG", value: "Google Search"])

type "google_start_page.searchbox", inputclick "google_start_page.googlesearch"

Tellurium: Testing New Blog Post...

Agile2009 - 86

2006-2

ui.UrlLink(uid: "create_new_post",clocator: [tag:'a', text: "New Blog Entry"])

ui.Form(uid: "blogform",clocator: [tag: 'form', name:'post'], group: "true") {

InputBox(uid: "title", clocator: [name: "title"])InputBox(uid: "content", clocator: [tag:'textarea', name: "content"])Selector(uid: "category", clocator: [name: "category"])Selector(uid: "author", clocator: [name: "author"])SubmitButton(uid: "post_button",

clocator: [name: 'btnPost', value: "Create Post"])}

ui.TextBox(uid: 'main_header', clocator: [tag: 'h1'])ui.TextBox(uid: 'category_header', clocator: [tag: 'h3', position: '2'])ui.TextBox(uid: 'author_header', clocator: [tag: 'h3', position: '3'])ui.Container(uid: 'table', clocator: [tag: 'table']) {

ui.TextBox(uid: 'content_para', locator: '//tr/td/p')}

...Tellurium: Testing New Blog Post

Agile2009 - 87

2006-2

openUrl "http://localhost:8080/"click "create_new_post"waitForPageToLoad 5000assert title == 'Welcome to SimpBlog'

// post blogtype "blogform.title", "Bart was here (and so was Tellurium)"selectByLabel "blogform.category", "Home"selectByLabel "blogform.author", "Bart"type "blogform.content", "Cowabunga Dude!"click "blogform.post_button"waitForPageToLoad 5000

// check contentsassert getText('main_header').matches('Post.*: Bart was here.*')assert getText('category_header') == 'Category: Home'assert getText('author_header') == 'Author: Bart'assert getText('table.content_para') == 'Cowabunga Dude!'shutDown

TrUMP IDE

Agile2009 - 88

2006-2

JWebUnit...

• Description– Java-based testing framework for web applications

– Intention is to provide a high-level "driver" Java API

– Wraps existing testing frameworks such as HtmlUnit

and Selenium with a unified, simple testing interface

– Support includes navigation via links, form entry and

submission, validation of table contents, and other

verification steps

– Includes some runner capabilities

– Useful in that it allows you to switch between

different lower level drivers without re-writing your

Agile2009 - 89

2006-2

...JWebUnit

Agile2009 - 90

2006-2

Source: http://jwebunit.sourceforge.net/

JWebUnit: Testing New Blog Post

Agile2009 - 91

2006-2

import net.sourceforge.jwebunit.junit.*

class TestSimpBlog extends WebTestCase {void setUp() { setBaseUrl("http://localhost:8080") }

void testPostBlog() {beginAt "/postForm"assertTitleEquals "Welcome to SimpBlog"setTextField "title", "Bart was here (and so was JWebUnit)"setTextField "content", "Cowabunga Dude!"selectOption "category", "Home"clickButtonWithText "Create Post"assert getElementByXPath('//H1').textContent.matches('Post.*: Bart was here.*')

def h3headings = getElementsByXPath('//H3')assert h3headings[1].textContent == "Category: Home"assert h3headings[2].textContent == "Author: Bart"def cell = getElementByXPath('//TABLE//TR/TD')assert cell.children[0].textContent == 'Cowabunga Dude!'

Agile2009 - 92

• Groovy Intro

Tellurium, JWebUnit

Test Runners– Native Groovy, JUnit, TestNG, Spock, EasyB, JBehave,

Cucumber

2006-2

Native Groovy

• Groovy has a friendly ‘==‘

• Built-in assert

• Scripts are low ceremony

• By utilising @Grab are easy to share

• Many in-built testing capabilities are

accessible even from scripts

• Easy to version control or treat like

operating system scripts

• Out of the box detection of JUnit and

TestNG tests

Agile2009 - 93

2006-2

Agile2009 - 94

2006-2

Built-in JUnit 3...

• Groovy distributions up to 1.6.X include JUnit 3

• Automatically invokes text runner

• Example uses HtmlUnit driverclass TestSimpBlogJUnit extends TestCase {

def page

void setUp() {page = new WebClient().getPage('http://localhost:8080/postForm'assert 'Welcome to SimpBlog' == page.titleText

void testBartWasHere() {// fill in blog entry and post itdef form = page.getFormByName('post')form.getInputByName('title').setValueAttribute('Bart was here (and so was form.getSelectByName('category').getOptions().find { it.text == form.getTextAreaByName('content').setText('Cowabunga Dude!')def result = form.getInputByName('btnPost').click()

...Built-in JUnit 3...

Agile2009 - 95

2006-2

...def form = page.getFormByName('post')form.getInputByName('title').setValueAttribute(

'Bart was here (and so was HtmlUnit)')form.getSelectByName('category').getOptions().find {

it.text == 'Home' }.setSelected(true)form.getTextAreaByName('content').setText('Cowabunga Dude!')def result = form.getInputByName('btnPost').click()

// check blog post detailsassert result.getElementsByTagName('h1').item(0).

textContent.matches('Post.*: Bart was here.*')def h3headings = result.getElementsByTagName('h3')assert h3headings.item(1).textContent == 'Category: Home'assert h3headings.item(2).textContent == 'Author: Bart'

// expecting: <table><tr><td><p>Cowabunga Dude!</p></td></tr></table>def cell = result.getByXPath('//TABLE//TR/TD')[0]def para = cell.getFirstChild()assert para.textContent == 'Cowabunga Dude!'

...Built-in JUnit 3

Agile2009 - 96

2006-2

GroovyTestCase Tests

• Like JUnit but with some enhancements– Additional assert methods

– fewer imports

– clean shouldFail syntax

Agile2009 - 97

2006-2

class TestSimpBlogGUnit extends GroovyTestCase {def page

void setUp() {// ...

void testBartWasHere() {// ...

Agile2009 - 98

2006-2

JUnit 4.X

import org.junit.*

class TestSimpBlogJUnit4 {def page

@Beforevoid setUp() {

// ...}

@Testvoid bartWasHere() {

// ...

• Groovy distributions from 1.7 include JUnit 4

• Automatically invokes text runner if needed

• Example uses HtmlUnit driver (not shown)

Agile2009 - 99

2006-2

JUnit 4.X Parameterized Tests

@RunWith(Parameterized)class TestSimpBlogJUnit4DD {

def page, author, title, category, content

TestSimpBlogJUnit4DD(author, title,category, content) {

this.author = authorthis.title = titlethis.category = categorythis.content = content

@Parameters static data() {return [

['Bart', 'Title 1', 'Home', 'Content 1'],['Homer', 'Title 2', 'Work', 'Content 2'],['Marge', 'Title 3', 'Food', 'Content 3']

].collect{ it as String[] }}

TestNG

Agile2009 - 100

2006-2

• Groovy automatically invokes text runner if run

as a script

• Example shows grouping, driver not shown

import org.testng.annotations.*

class TestSimpBlogTestNG {def page

@BeforeClassvoid setUp() {

// ...}

@Test(groups = [ "slow" ])void bartWasHere() {

// ...

TestNG Data Driven

Agile2009 - 101

2006-2

import org.testng.annotations.*

class TestSimpBlogTestNGDD {// ...

@DataProvider(name='SimpBlogDataProvider')Object[][] data() {

return [['Bart', 'Title 1', 'Home', 'Content 1'],['Homer', 'Title 2', 'Work', 'Content 2'],['Marge', 'Title 3', 'Food', 'Content 3']

].collect{ it as Object[] } as Object[]}

@Test(dataProvider = "SimpBlogDataProvider")void bartWasHere(author, title, category, content) {

// ...

Spock Testing Framework...

Agile2009 - 102

2006-2

...Spock Testing Framework

Agile2009 - 103

2006-2

• Testing framework for Java and Groovy

• Highly expressive specification language– No assertion API

– No record &

replay

mocking API

– No

superfluous

annotations

– Meaningful

assert error

messages

– Extensible

– Compatible

with JUnit

reportingwise

@Speck@RunWith(Sputnik)class PublisherSubscriberSpeck {def "events are received by all subscribers"() {def pub = new Publisher()def sub1 = Mock(Subscriber)def sub2 = Mock(Subscriber)pub.subscribers << sub1 << sub2

when:pub.send("event")

then:1 * sub1.receive("event")1 * sub2.receive("event")

Spock Example...

Agile2009 - 104

2006-2

import com.gargoylesoftware.htmlunit.WebClientimport spock.lang.*import org.junit.runner.RunWith

@Speck ()@RunWith (Sputnik)class TestSimpBlogSpock {

def page, subheadings, para, form, result

@Unroll("When #author posts a #category blog with content '#content' it should succeed"def "when creating a new blog entry"() {

given:page = new WebClient().getPage('http://localhost:8080/postForm')form = page.getFormByName('post')

when:form.getInputByName('title').setValueAttribute("$author was here (and so was Spock)"form.getSelectByName('category').getOptions().find { it.text == category }.form.getSelectByName('author').getOptions().find { it.text == author }.form.getTextAreaByName('content').setText(content)result = form.getInputByName('btnPost').click()subheadings = result.getElementsByTagName('h3')para = result.getByXPath('//TABLE//TR/TD/P')[0]

...Spock Example

Agile2009 - 105

2006-2

then:page.titleText == 'Welcome to SimpBlog'result.getElementsByTagName('h1').item(0).textContent.matches("Post.*: $author was here.*"subheadings.item(1).textContent == "Category: $category"subheadings.item(2).textContent == "Author: $author"

and:para.textContent == content

where:author << ['Bart', 'Homer', 'Lisa']category << ['Home', 'Work', 'Food']content << ['foo', 'bar', 'baz']

// Optional use of 'and:'

Agile2009 - 106

2006-2

• Description: BDD, Rspec-like testing librarynarrative 'segment flown', {

as_a 'frequent flyer'i_want 'to accrue rewards points for every segment I fly'so_that 'I can receive free flights for my dedication to the airline'

scenario 'segment flown', {given 'a frequent flyer with a rewards balance of 1500 points'when 'that flyer completes a segment worth 500 points'then 'that flyer has a new rewards balance of 2000 points'

scenario 'segment flown', {given 'a frequent flyer with a rewards balance of 1500 points', {

flyer = new FrequentFlyer(1500)}when 'that flyer completes a segment worth 500 points', {

flyer.fly(new Segment(500))}then 'that flyer has a new rewards balance of 2000 points', {

flyer.pointsBalance.shouldBe 2000}

EasyB Example ...

• When run will be marked as pending– perfect for ATDD

Agile2009 - 107

2006-2

scenario "Bart posts a new blog entry", {given "we are on the create blog entry page"when "I have entered 'Bart was here' as the title"and "I have entered 'Cowabunga Dude!' into the content"and "I have selected 'Home' as the category"and "I have selected 'Bart' as the author"and "I click the 'Create Post' button"then "I expect the entry to be posted"

...EasyB Example...

Agile2009 - 108

2006-2

description "Post Blog Entry Feature"

narrative "for feature", {as_a "Blogger"i_want "to be able to post a blog"so_that "I can keep others informed"

before "posting blog", {given "we are on the create blog entry page", {

webClient = new com.gargoylesoftware.htmlunit.WebClient()page = webClient.getPage('http://localhost:8080/postForm')

scenario "Bart was here blog", {

when "I have entered 'Bart was here' as the title", {form = page.getFormByName('post')form.getInputByName('title').setValueAttribute(

'Bart was here (and so was EasyB)')}

...EasyB Example...

Agile2009 - 109

2006-2

and "I have entered 'Cowabunga Dude!' into the content", {form.getTextAreaByName('content').setText('Cowabunga Dude!')

and "I have selected 'Home' as the category", {form.getSelectByName('category').getOptions().find { it.text == 'Home' }.setSelected(true

and "I click the 'Create Post' button", {result = form.getInputByName('btnPost').click()

then "I expect the entry to be posted", {// check blog post detailsassert result.getElementsByTagName('h1').item(0).textContent.matches('Post.*: Bart was here.*'def h3headings = result.getElementsByTagName('h3')assert h3headings.item(1).textContent == 'Category: Home' // traditional styleh3headings.item(2).textContent.shouldBe 'Author: Bart' // BDD style

// expecting: <table><tr><td><p>Cowabunga Dude!</p></td></tr></table>def cell = result.getByXPath('//TABLE//TR/TD')[0]def para = cell.firstChildassert para.textContent == 'Cowabunga Dude!'// para.shouldHave textContent: 'Cowabunga Dude!'

...EasyB Example...

Agile2009 - 110

2006-2

...EasyB Example

Agile2009 - 111

2006-2

2 scenarios (including 1 pending) executed successfully.

Story: simp blog initial

scenario Bart posts a new blog entry [PENDING]

given we are on the create blog entry page

when I have entered 'Bart was here' as the title

when I have entered 'Cowabunga Dude!' into the content [PENDING]

when I have selected 'Home' as the category [PENDING]

when I have selected 'Bart' as the author [PENDING]

when I click the 'Create Post' button [PENDING]

then I expect the entry to be posted [PENDING]

Story: simp blog

Post Blog Entry Feature

for feature

As a Blogger

I want to be able to post a blog

So that I can keep others informed

given we are on the create blog entry page

scenario Bart was here blog

when I have entered 'Bart was here' as the title

when I have entered 'Cowabunga Dude!' into the content

when I have selected 'Home' as the category

when I click the 'Create Post' button

then I expect the entry to be posted

easyb is preparing to process 2 file(s)

Running simp blog initial story (SimpBlogInitialStory.groovy)

Scenarios run: 1, Failures: 0, Pending: 1, Time elapsed: 1.049 sec

Running simp blog story (SimpBlogStory.groovy)

Scenarios run: 1, Failures: 0, Pending: 0, Time elapsed: 1.356 sec

2 total behaviors ran (including 1 pending behavior) with no failures

easyb execution passed

Cucumber• Description

– Loose coupling

between text spec

and step defns

Agile2009 - 112

2006-2

# language: enFeature: Addition

In order to avoid silly mistakesAs a math idiot I want to be told the sum of two numbers

Scenario Outline: Add two numbersGiven I have entered <input_1> into the calculatorAnd I have entered <input_2> into the calculatorWhen I press <button>Then the stored result should be <output>

Examples:| input_1 | input_2 | button | output || 20 | 30 | add | 50 || 2 | 5 | add | 7 || 0 | 40 | add | 40 |

# language: enFeature: DivisionIn order to avoid silly mistakesCashiers must be able to calculate a fraction

Scenario: Regular numbersGiven I have entered 3 into the calculatorAnd I have entered 2 into the calculatorWhen I press divideThen the stored result should be 1.5

Cucumber Example...

Agile2009 - 113

2006-2

# language: en

@newpost

Feature: New Blog Post

In order to create a new blog entry

Bloggers should be able to select their name and category and enter text

Scenario: New Posting

Given we are on the create blog entry page

When I have entered "Bart was here" as the title

And I have entered "Cowabunga Dude!" as the content

And I have selected "Home" from the "category" dropdown

And I have selected "Bart" from the "author" dropdown

And I click the 'Create Post' button

Then I should see a heading message matching "Post.*: Bart was here.*"

...Cucumber Example...

Agile2009 - 114

2006-2

...Cucumber Example

Agile2009 - 115

2006-2

import com.gargoylesoftware.htmlunit.WebClientthis.metaClass.mixin(cuke4duke.GroovyDsl)

Given ~/we are on the create blog entry page/, { ->page = new WebClient().getPage('http://localhost:8080/postForm')

When(~/I have entered "(.*)" as the title/) {String title ->form = page.getFormByName('post')form.getInputByName('title').setValueAttribute(title + ' (and so was Cucumber)')

When(~'I have entered "(.*)" as the content') {String content ->form.getTextAreaByName('content').setText(content)

When(~'I have selected "(.*)" from the "(.*)" dropdown') {String option, String name ->form.getSelectByName(name).getOptions().find {

it.text == option }.setSelected(true)}

When(~"I click the 'Create Post' button") { ->result = form.getInputByName('btnPost').click()

Then(~'I should see a heading message matching "(.*)"') {String pattern ->// ensureThat result.getElementsByTagName('h1').item(0).textContent.matches(pattern)

assert result.getElementsByTagName('h1').item(0).textContent.matches(pattern)}

Cucumber Data Driven Example...

Agile2009 - 116

2006-2

# language: en@newpostFeature: New Blog PostIn order to create a new blog entryBloggers should be able to select their name and category and enter text

Scenario Outline: New PostingGiven we are on the create blog entry pageWhen I have entered "<title>" as the titleAnd I have entered "<content>" as the contentAnd I have selected "<category>" from the "category" dropdownAnd I have selected "<author>" from the "author" dropdownAnd I click the 'Create Post' buttonThen I should see a heading message matching "Post.*: <title>.*"

Examples:| title | content | category | author || Title 1 | Content 1 | Home | Bart || Title 2 | Content 2 | Work | Homer || Title 3 | Content 3 | Food | Marge |

...Cucumber Data Driven Example

Agile2009 - 117

2006-2

JBehave

• Description– Behaviour-driven development in Java

• Also works out of the box for Groovy

– Behavior scenarios written in text

• Use the words Given, When, Then and And.

– Mapped using regular expressions and annotations

to step methods

– Web Runner available for non-technical users to

easily run tests

– Hooks to Selenium available in JBehave Web

• Other Java libraries (e.g. HtmlUnit) easy to use too

– Supports parameter converters

• Getting 'String' parameters into appropriate Object values

– Supports a 'StepDoc' function

• For listing available scenario clausesAgile2009 - 118

2006-2

JBehave Example...

Agile2009 - 119

2006-2

Given we are on the create blog entry pageWhen I have entered "Bart was here" as the titleAnd I have entered "Cowabunga Dude!" as the contentAnd I have selected "Home" from the "category" dropdownAnd I have selected "Bart" from the "author" dropdownAnd I click the 'Create Post' buttonThen I should see a heading message matching "Post.*: Bart was here.*"

import org.jbehave.scenario.Scenarioimport org.jbehave.scenario.steps.Steps

class NewPostScenario extends Scenario {NewPostScenario() {

super([new CreateBlogSteps()] as Steps[])}

}Scenario:

Given we are on the create blog entry page

When I have entered "Bart was here" as the title

And I have entered "Cowabunga Dude!" as the content

And I have selected "Home" from the "category" dropdown

And I have selected "Bart" from the "author" dropdown

And I click the 'Create Post' button

Then I should see a heading message matching "Post.*: Bart was here.*"

...JBehave Example...

Agile2009 - 120

2006-2

import org.jbehave.scenario.steps.Stepsimport org.jbehave.scenario.annotations.*import com.gargoylesoftware.htmlunit.WebClient

class CreateBlogSteps extends Steps {def page, form, result

@Given("we are on the create blog entry page")void gotoEntryPage() {

page = new WebClient().getPage('http://localhost:8080/postForm')}

@When('I have entered "$title" as the title')void enterTitle(String title) {

form = page.getFormByName('post')form.getInputByName('title').

setValueAttribute(title + ' (and so was JBehave)')}

@When('I have entered "$content" as the content')void enterContent(String content) {

form.getTextAreaByName('content').setText(content)}

Example uses HtmlUnit which must be added to CLASSPATH

...JBehave Example

Agile2009 - 121

2006-2

...@When('I have selected "$option" from the "$name" dropdown')void selectOption(String option, String name) {

form.getSelectByName(name).getOptions().find {it.text == option }.setSelected(true)

@When("I click the 'Create Post' button")void clickPostButton() {

result = form.getInputByName('btnPost').click()}

@Then('I should see a heading message matching "$message"')void checkPost(String pattern) {

assert result.getElementsByTagName('h1').item(0).textContent.matches(pattern)

...void checkPost(String pattern) {

ensureThat result.getElementsByTagName('h1').item(0).textContent.matches(pattern)

Choose either traditional

style or BDD style

JBehave Web Runner...

Agile2009 - 122

2006-2

...JBehave Web Runner

Agile2009 - 123

2006-2

Hacked* "JBehave Aware" GroovyConsole

Agile2009 - 124

2006-2

* Not currently publically available

Robot Framework...

• Description– Keyword-driven test automation framework

for acceptance level testing and acceptance

test-driven development (ATDD)

– Easy to use tabular syntax for creating test

– Easy to use test libraries implemented either with

Python or Java

– Open source, Apache License 2.0

– Supports creating data-driven test cases.

– Provides tagging to categorize and select test cases

to be executed

– Provides easy-to-read reports and logs in HTML

format

– XML-RPC interface for remote testing

Agile2009 - 125

2006-2

...Robot Framework...

• Tabular tests / Executable Specs– In HTML or Text

Agile2009 - 126

2006-2

***Settings***

Library OperatingSystem

***Variables***

${MESSAGE} Hello, world!

***Test Cases***

My Test [Documentation] Example test

Log ${MESSAGE}

My Keyword /tmp

Another Test

Should Be Equal ${MESSAGE} Hello, world!

***Keywords***

My Keyword [Arguments] ${path}

Directory Should Exist ${path}

• Reporting

Agile2009 - 127

2006-2

• Reporting

Agile2009 - 128

2006-2

• Tools

Agile2009 - 129

2006-2

Robot Framework Example...

Agile2009 - 130

2006-2

***Test Cases***

Submit postGiven we are on the create blog entry pageWhen I have entered "Bart was here" as the titleAnd I have entered "Cowabunga Dude!" as the contentAnd I have selected "Home" from the "category" dropdownAnd I have selected "Bart" from the "author" dropdownAnd I click the 'Create Post' buttonThen I should see a heading message matching "Post.*: Bart was here.*"

***Keywords***

Given we are on the create blog entry pageOpen Browser http://localhost:8080/postForm ${BROWSER}# Set Selenium Speed ${DELAY}Title Should Be Welcome to SimpBlog

When I have entered "${text}" as the titleInput Text title ${text}

First look at the Python version using Selenium

...Robot Framework Example

Agile2009 - 131

2006-2

And I have entered "${text}" as the contentInput Text content ${text}

And I have selected "${option}" from the "${list}" dropdownSelect From List ${list} ${option}

And I click the 'Create Post' buttonClick Button btnPost

Then I should see a heading message matching "${expected}"${text} = Get Text xpath=//h1Should Match Regexp ${text} ${expected}

***Settings***Library ${LIBRARY}Test Teardown Close Browser

***Variables***${BROWSER} *firefox3 C:/Program Files (x86)/Mozilla Firefox/firefox.exe${DELAY} 0${LIBRARY} SeleniumLibrary

Robot Framework with Groovy...

Agile2009 - 132

2006-2

***Settings***Library Remote localhost:8270 WITH NAME Groovy

***Test Case***Multiple Blog PostsPost Bart Home Title 1 Content 1Post Homer Work Title 2 Content 2Post Marge Food Title 3 Content 3

RunnerTestCase.txt

Groovy

XML RPC

Server

Groovy/Java

Driver

XML-RPC

...Robot Framework with Groovy...

Agile2009 - 133

2006-2

import groovy.net.xmlrpc.*import java.net.ServerSocketimport com.gargoylesoftware.htmlunit.WebClient

def server = new XMLRPCServer()server.get_keyword_names = { ["Post"] }server.get_keyword_documentation = { "" }server.get_keyword_arguments = { ["*args"] }server.run_keyword = { name, args ->

assert name == 'Post'def (author, category, title, content) = argsdef client = new WebClient()def page = client.getPage('http://localhost:8080/postForm')assert 'Welcome to SimpBlog' == page.titleTextdef form = page.getFormByName('post')form.getInputByName('title').setValueAttribute("$title (entered with Robot Framework)")form.getSelectByName('author').getOptions().find{ it.text == author }.setSelected(true)form.getSelectByName('category').getOptions().find{ it.text == category }.setSelected(true)form.getTextAreaByName('content').setText(content)def result = form.getInputByName('btnPost').click()assert result.getElementsByTagName('h1').item(0).textContent.matches("Post.*: $title.*")def h3headings = result.getElementsByTagName('h3')assert h3headings.item(1).textContent == "Category: $category"assert h3headings.item(2).textContent == "Author: $author"def para = result.getByXPath('//TABLE//TR/TD/P')[0]assert para.textContent == contentreturn [status:'PASS', output:"$name $args", error:'bad arg']

def serverSocket = new ServerSocket(8270)server.startServer(serverSocket)

In the future, a generic version of this file may be possible

...Robot Framework with Groovy...

Agile2009 - 134

2006-2

...Robot Framework with Groovy

Agile2009 - 135

2006-2

FitNesse/SLIM

• Description– Tool for enhancing collaboration

in software development

– Allows customers,

developers and

testers to easily

create and run

tests to compare

expected with

actual results

– Tests are captured

in wiki format and

mapped into code

using fixtures

Agile2009 - 136

2006-2

Source: http://fitnesse.org/

SLIM with Groovy...

Agile2009 - 137

2006-2

...SLIM with Groovy

Agile2009 - 138

2006-2

package simpblog

import com.gargoylesoftware.htmlunit.WebClient

class NewBlogPost {String author, title, content, categoryprivate resultdef execute() {

def client = new WebClient()def page = client.getPage('http://localhost:8080/postForm')assert 'Welcome to SimpBlog' == page.titleTextdef form = page.getFormByName('post')form.getInputByName('title').

setValueAttribute("$title (entered with Robot Framework)")form.getSelectByName('author').getOptions().find{

it.text == author }.setSelected(true)form.getSelectByName('category').getOptions().find{

it.text == category }.setSelected(true)form.getTextAreaByName('content').setText(content)result = form.getInputByName('btnPost').click()

}def mainHeading() {

def m = result.getElementsByTagName('h1').item(0).textContent =~/Post .*: (.*) $[^)]*$/

m[0][1]}

Example uses HtmlUnit to call SimpBlog web site but those details aren't important here

• Groovy Intro

Tellurium, JWebUnit

Non-web Drivers– SOAP/REST, Database, FEST, ...

Agile2009 - 139

2006-2

Better SQL Manipulation...

Agile2009 - 140

2006-2

import java.sql.Connection;import java.sql.DriverManager;import java.sql.Statement;import java.sql.ResultSet;import java.sql.SQLException;

public class JdbcJava {public static Connection getConnection() throws Exception {String driver = "org.hsqldb.jdbcDriver";String url = "jdbc:hsqldb:file:EMPDB";String username = "sa"; String password = "";Class.forName(driver);return DriverManager.getConnection(url, username, password);

public static void main(String[] args) {Connection conn = null;Statement stmt = null;ResultSet rs = null;try {conn = getConnection();stmt = conn.createStatement();String query ="select id, lastname, firstname from Employees";

rs = stmt.executeQuery(query);while (rs.next()) {System.out.println(rs.getString("id") +": " + rs.getString("firstname") +" " + rs.getString("lastname"));

}}catch (Exception e) {// handle the exceptione.printStackTrace();System.err.println(e.getMessage());

}// ...

// ...finally {// release database resourcesclose(rs); // close the ResultSetclose(stmt); // close the Statementclose(conn); // close the Connection

private static void close(ResultSet rs) {try {if (rs != null) rs.close();

} catch (SQLException e) {e.printStackTrace();

private static void close(Statement st) {try {if (st != null) st.close();

private static void close(Connection cn) {try {if (cn != null) cn.close();

...Better SQL Manipulation...

Agile2009 - 141

2006-2

import java.sql.Connection;import java.sql.DriverManager;import java.sql.Statement;import java.sql.ResultSet;import java.sql.SQLException;

public class JdbcJava {public static Connection getConnection() throws Exception {String driver = "org.hsqldb.jdbcDriver";String url = "jdbc:hsqldb:file:EMPDB";String username = "sa"; String password = "";Class.forName(driver);return DriverManager.getConnection(url, username, password);

public static void main(String[] args) {Connection conn = null;Statement stmt = null;ResultSet rs = null;try {conn = getConnection();stmt = conn.createStatement();String query ="select id, lastname, firstname from Employees";

rs = stmt.executeQuery(query);while (rs.next()) {System.out.println(rs.getString("id") +": " + rs.getString("firstname") +" " + rs.getString("lastname"));

}}catch (Exception e) {// handle the exceptione.printStackTrace();System.err.println(e.getMessage());

}// ...

// ...finally {// release database resourcesclose(rs); // close the ResultSetclose(stmt); // close the Statementclose(conn); // close the Connection

private static void close(ResultSet rs) {try {if (rs != null) rs.close();

private static void close(Statement st) {try {if (st != null) st.close();

private static void close(Connection cn) {try {if (cn != null) cn.close();

boilerplate

...Better SQL Manipulation

Agile2009 - 142

2006-2

import groovy.sql.Sql

def url = 'jdbc:hsqldb:file:EMPDB'def username = 'sa'def password = ''def driver = 'org.hsqldb.jdbcDriver'

def db = Sql.newInstance(url, username, password, driver)

db.eachRow("SELECT id, firstname, lastname FROM Employees") {println "$it.id: $it.firstname $it.lastname"

Agile2009 - 143

2006-2

More Details: Working with Databases

• Using standard SQL statements

• Using DataSets

def foo = 'cheese'def db = Sql.newInstance("jdbc:mysql://localhost:3306/mydb",

"user", "pswd", "com.mysql.jdbc.Driver")

db.eachRow("select * from FOOD where type=${foo}") {println "Gromit likes ${it.name}"

def db = Sql.newInstance("jdbc:mysql://localhost:3306/mydb","user", "pswd", "com.mysql.jdbc.Driver")

def food = db.dataSet('FOOD')def cheese = food.findAll { it.type == 'cheese' }cheese.each { println "Gromit likes ${it.name}" }

Agile2009 - 144

2006-2

WebTest testing Web Sitesdef ant = new AntBuilder()

def webtest_home = System.properties.'webtest.home'

ant.taskdef(resource:'webtest.taskdef') {

classpath {

pathelement(location:"$webtest_home/lib")

fileset(dir:"$webtest_home/lib", includes:"**/*.jar")

def config_map = [:]

['protocol','host','port','basepath','resultfile',

'resultpath', 'summary', 'saveresponse','defaultpropertytype'].each {

config_map[it] = System.properties['webtest.'+it]

ant.testSpec(name:'groovy: Test Groovy Scripting at creation time') {

config(config_map)

steps {

invoke(url:'linkpage.html')

for (i in 1..10) {

verifyText(description:"verify number ${i} is on pages", text:"${i}")

WebTest testing Emails

Agile2009 - 145

2006-2

def ant = new AntBuilder()

def webtest_home = System.properties.'webtest.home'

ant.taskdef(resource:'webtest.taskdef'){

classpath(){

pathelement(location:"$webtest_home/lib")

fileset(dir:"$webtest_home/lib", includes:"**/*.jar")

ant.testSpec(name:'Email Test'){

steps {

emailSetConfig(server:'localhost', password:'password',

username:'devteam@mycompany.org', type:'pop3')

emailStoreMessageId(subject:'/Build notification/',

property:'msg')

emailStoreHeader(property:'subject',

messageId:'#{msg}', headerName:'Subject')

groovy('''def subject = step.webtestProperties.subject

assert subject.startsWith('Build notification')''')

emailMessageContentFilter(messageId:'#{msg}')

verifyText(text:'Failed build')

SOAP Client and Server

Agile2009 - 146

2006-2

class MathService {double add(double a, double b) {

a + b}double square(double c) {

c * c}

import groovy.net.soap.SoapServer

def server = new SoapServer('localhost', 6789)server.setNode('MathService')server.start()

import groovy.net.soap.SoapClient

def math = new SoapClient('http://localhost:6789/MathServiceInterface?wsdl')

assert math.add(1.0, 2.0) == 3.0

assert math.square(3.0) == 9.0

FEST• Description

– Framework for testing Swing GUIs (among other things)

– Simulation of user interaction with a GUI (e.g. mouse / keyboard input)

– Reliable GUI component lookup• by type, by name or custom search criteria

– Support for all Swing components included in the JDK

– Compact and powerful API for creation and maintenance of functional

GUI tests

– Regular expression matching

– Supports Applet testing

– Ability to embed screenshots of failed GUI tests in HTML test reports

– Can be used with either TestNG or JUnit

– Supports testing violations of Swing's threading rules

– Experimental Groovy Builder support (coming soon!)

Agile2009 - 147

2006-2

dialog.comboBox("domain").select("Users")dialog.textBox("username").enterText("leia.organa")dialog.button("login").click()dialog.optionPane().requireErrorMessage()

.requireMessage("Please enter your .*")

• Groovy Intro

Tellurium, JWebUnit

Other Tools– SoapUI, ITest2, JMeter, Twist, ...

Agile2009 - 148

2006-2

Agile2009 - 149

2006-2

SoapUI...

...SoapUI

• Tool for testing Web Services has a built-

in Groovy editor for custom steps

Agile2009 - 150

2006-2

iTest2

• Tool for recording, refactoring and

running watir style tests

Agile2009 - 151

2006-2

We have had success cutting and pasting recorded test steps – using metaprogramming to map to Groovy testing DSL

Agile2009 - 152

2006-2

JMeter

• What is Apache JMeter?– Java desktop application designed to load test

functional behavior and measure performance

– Originally designed for testing Web Applications but

has since expanded to other test functions

Performance Testing

JMeter can call out to Groovy – reusing your functional tests

Agile2009 - 153

2006-2

Future version will support Groovy

• Groovy Intro

Tellurium, JWebUnit

Going beyond– Polyglot, Model-driven, Constraint/logic languages, Rules

Agile2009 - 154

2006-2

GParallelizer

Agile2009 - 155

2006-2

import java.util.concurrent.Future

// multiply numbers asynchronouslyAsynchronizer.withAsynchronizer(5) {

Collection<Future> result =[1, 2, 3, 4, 5].collectAsync{it * 10}

assertEquals(new HashSet([10, 20, 30, 40, 50]),new HashSet((Collection) result*.get()))

// run multiple closures in parallelAsynchronizer.withAsynchronizer {

assert AsyncInvokerUtil.doInParallel({calculateA()}, {calculateB()}

) == [10, 20]}

Constraint/Logic Programming...

Agile2009 - 156

2006-2

Source: http://xkcd.com/287/

...Constraint/Logic Programming

Agile2009 - 157

2006-2

Polyglot Programming

Agile2009 - 158

2006-2

ModelJUnit...

Agile2009 - 159

2006-2

...ModelJUnit

Agile2009 - 160

2006-2

More Information about Groovy

• Web sites– http://groovy.codehaus.org

– http://grails.codehaus.org

– http://pleac.sourceforge.net/pleac_groovy (many examples)

– http://www.asert.com.au/training/java/GV110.htm (workshop)

• Mailing list for users– user@groovy.codehaus.org

• Information portals– http://www.aboutgroovy.org

– http://www.groovyblogs.org

• Documentation (1000+ pages)– Getting Started Guide, User Guide, Developer Guide, Testing

Guide, Cookbook Examples, Advanced Usage Guide

• Books– Several to choose from ...

Agile2009 - 161

2006-2

More Information: Groovy in Action

Agile2009 - 162

2006-2

Second edition of GinA, ‘ReGinA’ now under development

groovy testing aug2009

Technology

how to make your testing more groovy

indigenous news lines au jul aug2009

eda presentation aug2009

groovy tutorial -...

agile testing pasadena jug aug2009

xpe admin guide aug2009

is groovy better for testing than java?

kalpataru power aug2009

members list aug2009

smart aug2009

testing storm components with groovy and spock

testing osgi the "groovy" way - lars pfannenschmidt, dennis...

step by step aug2009

master thesis charlottesteffensen aug2009

svcc groovy testing

introduction · 11 testing with groovy in this chapter, we...

ilomo clampi botnet aug2009

approaching gradle 3 - jug saxony testing of your build...

deshielo evaporadores industriales aug2009

gtac boosting your testing productivity with groovy