continous ui testing with espresso and jenkins

Continous UI testingby

with Espresso and Jenkins

Sylwester MadejFounder & Head Developer@smdremedy

#droidconZG

sylwester.madej@byoutline.com

byoutline.com

2

This is a story of my team's journey

...jurney to CI/CD

We are a small team

...jurney to CI/CD

We are a small team

- we are looking for things to be cost effective- not a perfect solution, but a decent one

What we have gained

- better, more agile work flow- no more Commit and run situations- way to check UI on many configurations- less regression bugs- higher code quality

We started with tests

- JVM tests (finally works in AS)- Robolectric- Instrumentation

We wanted more from VCS

- Use DVCS- Gitflow = git++- Branch per feature, tag per release

Static code analysis

- FindBugs- PMD- Checkstyle- SonarCube

Continous Integration checklist

- gitflow ✔- unit tests ✔- code coverage ✔- static analysis ✔- build with one command ✔

We needed our CI server

- Jenkins- Travis- Circle CI- TeamCity- Bamboo...

Why Jenkins?

- free of change, but not of hustle- great community- can be hosted on any device- hundreds of plugins

Where to host it?

- CloudBees.com- VPS: s3, digital ocean- bare metal

Perfect local Jenkins machine

- good price/performance (i5, ssd, ram)- energy effective- quiet- small

Candidates:- Intel NUC (7W idle)- Mac Mini

CI setup

Ubuntu- apt-get install- sdk install (Android Plugin don't install everything)- kvm- plugins (Chuck Norris, Lint Publisher, jacoco...)

First easy win

- create new project- add git with credentials

./gradlew build test jacoco

You get:- compilation check- lint check- unit tests with code coverage

We wanted more

Instrumentation tests- Robotium- Appium- Espresso- UI Automator

Some Espresso

Great way to test UI, and do integration testsApi from developers to developers

onView(Matcher).perform(ViewActions).check(ViewAssertions)

More on Maciej's speach at 12:00:„Expressive functional testing with Espresso”

How to test API?

Mock/stub API classes

- Dagger FTW- works great with Retrofit

How to test API?

MockWebServer

- set HTTP mock for one test- Square Quality™, build upon OkHttp- mock calls, verify where they called

How to test API?

AndroidStubServer

- HTTP server build in your APK (good for integration testing )- can be sent to client within app

simple config:{"method": "GET",

"path": "/books", "response file": "books.json"}

https://github.com/byoutline/AndroidStubServer

Where to run Instrumentation tests?

- androd emulators (x86 if possible)- genymotion- real devices

Start simple

- create job with single emu- use Android Emulator Plugin for configuration- start instrumentation tests in gradle

./gradlew connectedCheck

Enter the matrix

- create Multi-configuration project- exclude combinations

Need a Spoon for your Espresso?

- great tool from Square (duh !)- run instrumentation tests on multiple devices in parallel- external plugin to run with gradle

https://github.com/stanfy/spoon-gradle-plugin

- connect your devices to CI server with USB hub- generates nice looking reports

One step further - Continous Delivery

- your code is tested and ready to meet the world- create new job to deploy to distibution channel

CrashlyticsGoogle Play Alpha/Beta

- start after succesful build or manualy

When we need more devices

Devices in the cloud

- AppThwack - TestDroid...

Bonus:Some free Intel devices

How to make it even better?

Genymotion on server

http://www.slideshare.net/vishyrich/genymotion-with-jenkins

BDD with Calabash

Black box testing

calabash-android genwrite specification with Gherkin DSL Scenario: As a valid user I can log into my app When take picture When I press "Login" Then I see "Welcome to coolest app ever" Then take picture

calabash-android run APK

Works great with AndroidStubServer!

Sylwester MadejFounder & Head Developer@smdremedy

sylwester.madej@byoutline.com

byoutline.com

30