How to Build an Appium Continuous Testing Pipeline

Step-by-Step TutorialNovember, 2017

Today’s speakers

Guy Arieli,

CTO,

Experitest

Ruth Zamir

Marketing Director

Experitest

Agenda

3

Why do we need continuous testing?

The continuous testing environment

Step-by-step demo

01

02

03

Q&A04

15 min

30 min

5 min

10 min

Digital transformation impacts business

4

Frequent app releases (mobile: 2 weeks, web: daily,

even hourly)

Digital is a keysurvival strategy

Customer empowerment

R&D organizations under pressure

more, faster, better

Unique mobile market characteristics

• Fast technology innovation & complex

integration (Alexa/Siri, IoT, AR..

• Device matrix large and constantly evolving

• High cost of error - App/play stores create a

distribution gap that makes rollback impossible

5

Scope * Velocity * Risk

SDLC Paradigm shift

Speed, agility and risk reduction

6

Agile CI/CDDevOps

“Develop smaller chunks - we will probably change our mind”

“get your team talking…”

“make sure you didn’t break anything…”

What is Continuous Integration?

“a software development practice where members of a team integrate their work frequently...

Each integration is verified by an automated build… to detect integration errors as quickly as possible.”

There is no Continuous Integration without Continuous Testing

The estimated* impact of a shortened feedback loop

R&D productivity will increase from 10% to 35% since more time will be invested

on Dev-test activities

Dev-test

10%

Project

related

15%

Context switching

20%Bug reproduction

25%

Root cause analysis

25%

Administration

5%

R&D productivity - 1 week feedback

Dev-test

35%

Project

related

20%

Context

switching

14%

Bug reproduction

13%

Root cause analysis

13%

Administration

5%

R&D productivity - 1 hour feedback

* Data source: Experitest customers

Rapid feedback significantly increases R&D effectiveness

0

10

20

30

40

50

60

1 week 1 day 6 hours 2 hours 1 hour 1 minute

Time to feedback

R&D Effectiveness

The dev-test process

Epic Story thinning PrototypeAcceptance

Criteria

Code (incl. White box Test)

Build & DeployAutomated

testingAcceptance

testing

Showcase / Demo Sign-off

Dev Engineer

Dev & Test EngineerDevOps EngineerDev & Test Engineer

Functional Spec

Product Owner

Analyst Product Owner

Analyst Designer Analyst Tester Test Engineer

TesterProduct Owner

CI/CD cycle

12

High scales simultaneous testing of different use cases, devices and network conditions

Highly stable test suites

Test environment management requires IT / Security expert

Rapid result analysis and actionable feedback

Continuous testing in mobile - implications

Agenda

13

Why do we need continuous testing?

The continuous testing environment

Step-by-step demo

01

02

03

Q&A04

Development Engineer

The Continuous Testing process

Commit tests

Trigger test execution

Load app to cloud

Test Engineer

Commit app code

Consolidate test results

Continuous Integration

reporttest

build Version approved?

no yes

Debug / update test suite

1Pull app source2

Build app3

Pull test source4

Build test project5

6 7 8

9

10

Trigger build

Today’s demo environment

• Appium tests

• TestNG framework

• Jenkins server

Pre-prepared

• Appium* Studio for Eclipse

• Eclipse*

• Jenkins*

• Git

• Gradle

• Experitest Digital Device Lab

• Experitest Reporter

Tools used

• Build Application (Jenkins)

• Pull tests from Git and build test project (Jenkins + Gradle)

• Upload application to the cloud - Jenkins (shell script)

• Run Tests (Jenkins + Digital Device Lab)

• Link test results to Jenkins

What we will do

* Any test framework, IDE and CI tools can be used

Agenda

16

Why do we need continuous testing?

The continuous testing environment

Step-by-step demo

01

02

03

Q&A04

Configuration and setup

17

Demo

Executing from Jenkins

18

Recap

• Appium*

• TestNG*

• Eclipse*

• Jenkins*

• Git

• Gradle

• Appium Studio for Eclipse

• Experitest Digital Device Lab

• Experitest reporter

Tools / frameworks used

• Used CI server to build an applicationfrom repository

• Used CI server to build a test project from repository

• Used CI to upload application to digital device lab

• Used CI and digital device lab to installed application numerous devices and run tests in parallel on

• Used Reporter to link test results to CIand analyzed test results

What we did

* Any test framework, IDE and CI tools can be used

How the SeeTest platform support hands free continuous testing

20

Epic Story thinning PrototypeAcceptance

Criteria

Code (incl. White box Test)

Build & DeployAutomated

testingAcceptance

testing

Showcase / Demo Sign-off

Dev Engineer

DevOps EngineerDev & Test Engineer

Functional Spec

Product Owner

Analyst Product Owner

Analyst Designer Analyst Tester Test Engineer

TesterProduct Owner

Dev & Test Eng.

Digital assurance Lab (SeeTestCloud)

Grid Execution

Test development

Test Analytics

Manual Testing Grid Execution Dev &

DebugTest Analytics Test Analytics

RESTful API to mass

deploy application on

targeted devices

Dev-Test Feedback cycle – 1 hour

Agenda

21

Why do we need continuous testing?

The continuous testing environment

Step-by-step demo

01

02

03

Q&A04

Thank You!

22