behavior driven development by example

BDD – By ExamplesBehavior Driven Development with Selenium/ Cucumber-JVM

Nalin Goonawardana

The audience

Automated acceptance testing? Agile/Scrum? CI? BDD?

BDD/BDT – Connecting the dots

Features

Agile

Gherkin

Cucumber

UATDD

TDD

BDD-2003

Selenium-2004

Jason HugginsJBehave

Thought Works

Dan North

Cucumber-JVM

Step Definitions

Steps

User Stories

Specification by

Example

Scenarios

DSL

Business language --- DSL Specification by Example On the other hand a good functional test tells a story!

The concept of “Xdriven”

You are driven by something to do something or achieve something. i.e. What do you set as the guiding line that governs how you

establish some other task! Few examples: Capability driven development Model driven development Test driven development Passion driven development Requirement driven development? What drives us to do something or what you do first! Of course it is not “Bug driven development where the developer

only get to know the expected requirement through a bug!

Module

How close we are - Requirements

Class

Unit?-method?

User Interface

Unit testing

Module level testing/Unit testing

Integration testing

Service layer testing

Functional Testing

User acceptance Testing

User

Developer

Expected behavior

The creation of unit tests and "technical" code

The creation of functional tests and "features"

What is wrong with Unit tests

Developer testing Vs. Behavior compliance.

RequirementOr Expected behavior

Verification and validation at unit test level

Developer

BA/QA/User

Explicit collaboration effort that May vary

Gaps and Cracks

Bugs

One common set of verifications that collaboratively making passedas the exit criteria or done-ness criteria

The gap between unit tests : high-level software requirements, low-level technical details

Behavior Requirement

Gaps and Cracks Contd..

Executable Specification - User stories

Executable specifications Apply the "Five Why's" principle to each proposed User Story, so

that its purpose is clearly related to business outcomes Thinking "from the outside in", in other words implement only

those behaviors which contribute most directly to these business outcomes, so as to minimize waste

Describe behaviors in a single notation which is directly accessible to domain experts, testers and developers, so as to improve communication

Apply these techniques all the way down to the lowest levels of abstraction of the software, paying particular attention to the distribution of behavior, so that evolution remains cheap

What makes BDD different?

It technically Integrates the requirement to the SDLC. It demands developer to align the development with user

requirement from the beginning. It is a engineering work bench that support requirement driven

deveolopment.

BDD and BDT

How do we implement this?

Process or workflow alignment Technical solution

The work-flow

Feature/Behavior Acceptance

Feature > Scenarios

Test Fixtures

App implementation

BDD phases

Elicitation – BA/Customer (Product)

Validation(Exploratory+BDT)

Elaboration – BA/QA (Product/QA)

Acceptance(Product owner-Demo)

Implementation of feature - Dev Implementation of steps - QA

The framework anatomy

Page One Page Two Page Three

get_element1

action_element1 action_element2

get_element2

Page Object layer (Pages and Elements)

Basic Action Layer

Composite action Layer (Helper abstract layer-Business flow layer)

Exiting comp Action

Intermediate Comp Action

Starting comp action

[Transferring browser control over page objects]

Step Definition Layer (Collection of methods annotated in Cucumber/Gherkin - Given, When, Then, But or And)

Feature Layer (Actual test cases represented as scenarios written in annotated names in step definition class.)

Junit executor class

Page Object layer (Pages and Elements)

The framework anatomy contd..

Basic Action Layer

Composite action Layer (Helper abstract layer-Business flow layer)

Step Definition Layer (Collection of methods annotated in

Cucumber/Gherkin - Given, When, Then, But or And)

Feature Layer (Actual test cases represented as scenarios written in

annotated names in step definition class.)

TDD process and it’s roots

Write

little test

Watch

test fail

Refactor

Get test pass

Wire steps with

automation code

Implement the App

complying to the

Behaviour

Specify Behavio

ur

Into details

Cucumber Gherkin

The jargon

Feature : Piece of system functionality that delivers value. User story : Vertical functional slice of a feature that can be

delivered independently Scenario : One key example for a feature in the feature file. This

amounts to the main functional flows/ behaviors * critical data (derived through a test design technique)

Step: Domain language phrases which can be combined to create scenarios.

Cucumber

Cucumber is a testing framework that speaks the language of Gherkin.

It wires the .feature or the executable spec to its corresponding test implementation the “step definitions”.

Using Cucumber doesnt mean you are practicing BDD!

Cucumber-JVM and the tech stack

• Cucumber-JVM is a pure Java implementation of Cucumber, with native support for the most popular JVM languages: Java, Scala, Groovy, Clojure, Rhino, Jython and JRuby.

• Selenium Web Driver : Java : Junit : Maven

Gherkin in two mins

Gherkin is the language that Cucumber understands. purposes — documentation and automated tests it’s talking to you, telling you what code you should write. Single Gherkin source file contains a description of a single

feature. Source files have .feature extension.

Gherkin in two mins …..

Features,User Stories, Scenarios and steps line-oriented language that uses indentation to define structure Line endings terminate statements Either spaces or tabs may be used for indentation lines start with a keyword

(Feature,Background,Scenario,Scenario Template,Given,When,Then,And,But)

Comment lines are allowed anywhere # Regular expressions Tables

Gherkin in two mins ….. Keywords

Given : Pre condition When: primary action of a scenario Then : Post condition, expected observable outcomes. And, But

If you have several Given, When or Then steps.

Data driven

Scenario Outline: Login Success and Failure Given I navigate to the mock application When I try to login with '<type>' credentials Then I should see that I logged in '<status>'

Examples: | type | status | | valid | successfully | | invalid | unsuccessfully|

Setting up the IDE

Pre conditions: IntelliJ IDEA 12 Community Edition

In the IDE add the following plugins;Settings> IDE settings > Plugins > Install JetBrains Plugins > Gherkin V1.1Settings> IDE settings > Plugins > Install JetBrains Plugins > Cucumber for Groovy V1.0Settings> IDE settings > Plugins > Install JetBrains Plugins > Cucumber for Java V1.0

IDE features

1. Code Completion (Step definition and Gherking syntax support)

2. Cucumber Framework Support 3. Drill down from *.feature file to step definition 4. Code Formatting 5. Junit runner support

Framework Changes Step 1 – Adding Maven Dependencies

Simple five steps approach in the framework level changes Assumptions Existing Java/Junit/Maven/Web Driver based test framework. Or Create an empty maven project Apart from the junit/Selenium deps add the following deps to

the maven pom file <dependency> <groupId>info.cukes</groupId> <artifactId>cucumber-java</artifactId> <version>1.1.4</version> <scope>test</scope> </dependency> <dependency> <groupId>info.cukes</groupId> <artifactId>cucumber-junit</artifactId> <version>1.1.5</version> <scope>test</scope> </dependency>

Framework Changes Step 2 - Set up package structure

Framework Changes Step 3: Add the junit runner class

Framework Changes Step 4 - Add the step definition class

Framework Changes Step 5 - Add the feature file

The role of the CI dashboard

Create a view in the name of the sprint and add the junit runner class to separate new projects and let every one know where we are!

Demo

Sprint X starts Scenarios were written Create CI project dashboard for Sprint X and initiate. QA : Write the test fixture and implement any step definitions.

Use mock elements(xpath or id etc.) refering to the wire frames and create the page objects. Replace them with the real element references as soon the details available.

Dev: Implement the Code Tests getting passed progressively Done: All tests are passed + Planned manual tests are

executed.

Auto generated code skeleton

@Given("^add (\\d+) and (\\d+)$")

public void add_and(int arg1, int arg2) throws Throwable {

// Express the Regexp above with the code you wish you had

throw new PendingException();

}

The transition

In the original proposal the transition would be just using “should” instead of “test” in the unit testing!

User story: "As a [role] I want [feature] so that [benefit]". Scenario: “Given [initial context], when [event occurs], then

[ensure some outcomes]” Test method names should be sentences A simple sentence template keeps test methods focused An expressive test name is helpful when a test fails “Behavior” is a more useful word than “test” Requirements are behavior,too Acceptance criteria should be executable

Agile Scrum Alignment

Feature : Backlog item Scenario: Test case Donness : Built in - Green report + Manual expert/exploratory

testing sign off. Continuous collaboration : BDD enforces it. Continuous integration : BDD enforces it. (Part of the build.) Recommends automation : BDD enforces it. Just enough/living documentation that serve multiple purposes. Always know how much is remaining.

Manual testing?

Expertize driven testing Exploratory testing Regression ? Limitations of automation ?

Can we only do BDT without BDD

Yes and a No! BDT can be identified as the means of realizing BDD.

Some more advantages

Non technical BA's Product owners, OPS, Dev, QA speaking the same

language Defines Doneness criteria Automated Regression Test suites. No compilation issues only runtime. User expected behavior conformance is maintained from the

beginning...

Advantages Contd…

Tests can become truly useful documentation Helps you to separate "what?" from "how?" Purposely small command-set is simple to learn Separating requirements from implementation helps refactoring Specifications can be as thorough as unit tests Specifications can be hooked in at different levels Agreeing details up-front allows finer control over scope Test coverage no longer relies on developer self-discipline automatic generation of technical and end user documentation

from BDD "specifications"

The collaboration

• In BDD a significant portion of "functional documentation" defined in the form of “User Stories”.

• "tests", a BDD practitioner will prefer the terms "scenario" and "specification".

• "functional tests", the preferred term will be "specifications of the product's behavior".

• "the unit tests of a class", a practitioner or a team using BDD prefers to speak of "the specifications of the behavior of the class"

You are already a BDD practitioner

If you are practicing TDD properly BDD, often described as “TDD done well” User focused TDD

TOOLS

JSpec - JavaScript Behavior Driven Development SpecFlow - Pragmatic BDD for .NET

• FitNesse• Cucumber (software)• Framework for Integrated Test• JBehave• Robot Framework• Concordion• specs2• specflow

BDT is a paradigm shift for QA

• Reclaiming the QA role within the Agile world

• No tests but scenarios!

• Technical convergence

• Early/Parallel automation.

• Just enough Engineering practices!

• Tests frameworks without layers ! Record and play !

• Highly engaged requirement deal

• Its not about the tools but about the practices !

Future implementations

API Testing (In conjunction with Rest Assured)

Performance testing (In conjunction with Jmeter)

Consolidated online CASE tool?

A communication and collaboration framework for developers, QA and non-technical or business participants in a software project.

More

• http://aslakhellesoy.com/post/20006051268/cucumber-jvm-1-0-0

• http://www.ibm.com/developerworks/opensource/library/a-automating-ria/index.html

• RSpec, Cucumber and GivWenZen(Fitness)

• FitNess

• http://fitnesse.org/

• Xebium : Combine the power of Selenium

• http://xebia.github.io/Xebium/

• BDD original proposal

• http://dannorth.net/introducing-bdd/

Q&A

Thanks!

Nalin Goonawardanahttp://nalingski.blogspot.com/

Leapset Inc.http://www.leapset.com/

http://www.meetup.com/colombo-selenium/events/165499102/