Continuous TestingDigital Transformation Requires Continuous Testing

Ingo Philipp

Businesses must continuously exploit

digital technologies toboth create new sourcesof customer value and

increase operationalagility in service of

customers.

© Forrester Research

DigitalDisruption

Across industries, companies face thechallenge of software-led transformation

Plan Code Version Build Provision Test Deploy Monitor

© Melissa Perri

IdentifyNeeds

ReceiveMoney

BuildSolution

Your Company Your Customer

Requirements Design Implementation Testing Acceptance Deployment

Inception Elaboration Construction Transition

Development Operations

DevOps

Agile Horizon

Digital Singularity

IncrementalRational Unified Process

4

AgileScrum, Kanban

2

ContinuousDevOps

1

6+

Big BangWaterfall

Development Response

Flexible Framework

Rigid Rules

Following a Plan

Contract Negotiation

Copious Documentation

Processes & Tools

Individuals & Interactions

Working Software

Customer Collaboration

Responding to Change

It’s not the strongestthat survive, nor the most

intelligent, but the one mostresponsive to change

Charles Darwin

100%

0%

SprintStart

SprintEnd

API

UI

Start testing early,shift left test automation Degree of

Completion

Time

Efficiency

Gain

+4xCreation

+6xMaintenance

+20xExecution

*voke, market snapshot report on service virtualization - 2012

On average, organizations require access to 33 systems for development or testing.

18Average # of systems with

unrestricted access

96% of testers have

restricted test lab access

Testing

Present

Enterprise system landscapes are alike disease gene networks.

Welcome to the tester’s hell

Automation doesn't make testingeasy, it makes testing possible

© Wolfgang Platz

Lesson Learned

A

B

Visiting London

~100 Billion Billion Billion Billion Billion Billion Billion Combinations= 50

Testing harder isn’t theanswer, testing smarter is!

© Wolfgang Platz

Lesson Learned

© BBC

Testing is exactly likewashing a pig. Because

it’s messy. It has no rules. Noclear beginning, middle, or end.

It’s kind of a pain in the ass, and when you’re done you’re

not sure if the pig is really clean or even why you were washing

a pig in the first place.

© Luke Sullivan © BBC

Testing is exactly likewashing a pig. Because

it’s messy. It has no rules. Noclear beginning, middle, or end.

It’s kind of a pain in the ass, and when you’re done you’re

not sure if the pig is really clean or even why you were washing

a pig in the first place.

© Luke Sullivan

That’s what we can know

Testing is exactly likewashing a pig. Because

it’s messy. It has no rules. Noclear beginning, middle, or end.

It’s kind of a pain in the ass, and when you’re done you’re

not sure if the pig is really clean or even why you were washing

a pig in the first place.

© Luke Sullivan

That’s what we can know

That’s what we know

That’swhat wecheck

Problem

Knowledge Gap

© Richard Bradshaw

© Cem Kaner

© Richard Bradshaw

The purpose of testing is toclose the knowledge gap

The goal is information,not gratuitous automation

Testing is and always will bea search for information

That’s what we can know

That’s what we know

That’swhat wecheck

Problem

Knowledge Gap

Test Case

PassingTest Case

PassingTest Case

@iamdevloper

PassingTest Case

@iamdevloper

Checking« Requires Processing »

© Rich Rogers

Verify through

InstructionsPay attention to

Deviations

Create

Test CasesFollow

Procedure

Examine

RequirementsFactory

Process

PassingTest Case

@iamdevloper

» Here You Monitor Known Risks

Confirmatory Testing« Requires Processing »

© Rich Rogers

Verify through

InstructionsPay attention to

Deviations

Create

Test CasesFollow

Procedure

Examine

RequirementsFactory

Process

Me

ch

an

ica

l P

roce

ss

Confirmatory Testing« Requires Processing »

© Rich Rogers

Verify through

InstructionsPay attention to

Deviations

Create

Test CasesFollow

Procedure

Examine

RequirementsFactory

Process

Exploratory Testing« Requires Thinking »

Investigate through

ExperimentsPay attention to

Oracles

Create

Test IdeasFollow

Clues

Examine

RisksAdaptive

Investigation

» Here You Analyze Potential RisksC

og

nitiv

e P

roce

ssM

ec

ha

nic

al

Pro

cess

» Here You Monitor Known Risks

CheckingEfficient Confirmatory Testing

ExploringEffective Exploratory Testing

TestingThorough Testing

+

=

AgileTesting Equation

© Elisabeth Hendrickson

100%

ManualTesting

0%

AutomatedTesting

TestingPresent

There’s a way to do it better. Find it!

Project Charter

11Manual

Testers

4755Manual

Test Cases

?Unknown

Coverage

10Weeks

Execution

Risk Coverage Optimization Test Data Management UI/API Test Automation Continuous Integration & Distributed Execution

48Hours

Execution

Single Agent

8Hours

Execution

Multiple Agents

11Manual

Testers

4755Manual

Test Cases

?Unknown

Coverage

10Weeks

Execution

89%Business Risk

Coverage

1193Manual

Test Cases

37%Business Risk

5Weeks

Execution

2.5Weeks

Execution

92%Automated

Test Cases

53%Business Risk

Coverage

Smoke Testing

34Minutes

Execution

Smoke Testing

Test Case Count 7%

75% Redundancy 50% Effort Test Data

64% API Tests

8Hours

Execution

Multiple Agents

72%Critical Defect Reduction

Production Defects

89%Business Risk

Coverage

1193Manual

Test Cases

5Weeks

Execution

2.5Weeks

Execution

92%Automated

Test Cases

34Minutes

Execution

Smoke Testing

53%Business Risk

Coverage

Smoke Testing

Test Case Count 7%

Test Case Design & Exploratory Testing

64% API Tests

Risk Coverage Optimization Test Data Management UI/API Test Automation Continuous Integration & Distributed Execution

The future doesn’t just happen,it gets happened, so make it happen

© Paul Tagliablue

Conclusion

QuestionsThe show is over. It's your turn.

?