1

Packaged vs. Custom Application Testing

Shoeb Javed, CTO

September 07, 2017

2

Agenda

• Digital Drivers

• Packaged vs. Custom Apps

• Prioritizing Testing for Packaged

Apps

• Rethinking Automation

• Questions

Presented by Shoeb JavedWorksoft CTO

3

“It is not the strongest of the species

that survives, nor the most intelligent

that survives. It is the one that is the

most adaptable to change”

- Charles Darwin

4

“More than half of the names of companies on

the Fortune 500 have disappeared since the

year 2000.” 1

Disrupt or Be Disrupted!

Don’t Know What Their Industry

will Look Like in 3 Years248%

Feel They May Become

Obsolete in 5 to 10 Years245%

Feel Threatened by Digital

Start-Ups278%

1World Economic Forum2 Research by Vanson Bourne & Dell Technologies

5

26

Enterprise Apps Are Core to Digital Transformation

Chart Source: Mckinsey, http://www.mckinsey.com/business-functions/digital-mckinsey/our-insights/the-case-for-digital-reinvention1 2017 ERP Report, Panorama Consulting

“As postmodern ERP becomes

pervasive, many organizations

will face increasing complexity,

risk and cost that they had not

foreseen and are not equipped

to deal with.”

- Gartner, Transforming ERP to

Postmodern ERP Primer for 2016

Of ERP projects are

viewed as failures.1

Of ERP projects

exceed their budgets.1

But Change is Increasingly Complex

26%

74%

6

Why Do We Need to

Think Differently

About Enterprise

Applications?

7 9/12/2017 © 2017 Worksoft, Inc.

Today’s Modern Enterprise Applications. Houses or Cities?

8

Design (Fit/Gap)

Requirements

SDLC is Different

• Type of work

• Project scale

• Scope of application functionality

• Size of the “foundation” to be laid

• Size of minimal unit of functionality providing business value

• Potential impact of production deployment failures

Build (configuration, master data,

write code; integrate entire business

processes; unit test)

Test

Functional, Regression,

Integration, UAT, Performance

Deploy to Production

Custom

Development

Build (write code, unit

tests)

Developer Test

Deploy

Requirements

Architect

Integrate

UI, backend

TestFunctional, Regression, UAT

Deploy

Packaged

Applications

9

Test Automation Considerations

Understanding Complexity UI Driven

10

Understanding

11

Complexity

12

The UI is Critical.!It’s what the world sees!

What happens

when we don’t!

End-to-End System Tests and User

Acceptance Tests are run at the GUI level.

It’s what the world sees and validates whether or not

the business process actually works.

13

Business Analysts/QA

“Does my Business Work?”

Developers/Test Engineers

“Does my Software Work?”

UI Tests – The Bulk of Packaged App Tests

13

End to End (UI)

Integration ( API)

Unit (Code)

. UI Testing Remains the Most Challenging to

Automate!

14

Mainframe Client Server Web 1.0 Web 2.0 / Mobile HTML 5 VR & Biometrics

1980 2000

1990

2020

2010 2030

Modern Day UI EvolutionAs the way applications are written changes, so must the way we test.

15

Example. What’s Your UI? Fiori. Ok. But Which One?

SAP Fiori Cloud DemoSAPUI5

v 1.26 – 1.28

Image source:

https://demo-

fioritrial.dispatcher.hana.ondemand.com/sites

S/4HANA 1511 SAPUI5 v 1.30.x or higher

(Fiori 1.0)

Image source:

https://experience.sap.com/fiori-design-

web/sap-fiori/#sap-fiori-10

S/4HANA 1610 SAPUI5 v 1.40.x or higher

(Fiori 2.0)

Image source:

https://experience.sap.com/fiori-design-

web/sap-fiori/#sap-fiori-20

16

How Do We Prioritize Test Automation?

17

Prioritization Starts with the Business!

“Quality and testing in today’s fast-moving

continuous delivery environment is quickly

becoming everyone’s responsibility, so

management need to ensure their people are

properly enabled.”

- Freeform Dynamics, Continuous Testing as a Digital Business

Enabler Research Report, 2017

86%Of testing leaders report they

need to work closer with

business leadership!1

1PAC Group, Digital Testing in Europe: Strategies, Challenges, & Measuring Success, 2015

OperationsEasy to Scale

QAEasy to Use

Easy to Maintain

Application

Team (IT)Easy to Integrate

BusinessEasy to Use

18

Late Defect Discovery is Expensive and Disruptive!

0

9599

22781331

$18 ($0)

$180 ($1.7 M)

$360 ($820K)

$1,800 ($2.3 M)

$0

$200

$400

$600

$800

$1,000

$1,200

$1,400

$1,600

$1,800

$2,000

0

2,000

4,000

6,000

8,000

10,000

12,000

Discovered inRequirements/Specs

Discovered in TestExecution

Discovered in UAT Discovered inProduction

Defect Correction Cost (5 Project Sample)

Defects Cost to correct per defect

19

03

02

01

DEPLOY AT AN

INDUSTRIAL SCALE

IN THE CLOUD

FILTER BASED

ON RISK

BUILD AUTOMATION

FASTER

Pre-built content and

optimizations for popular

enterprise applications.

Automatic test generation

from automated discovery.

Intelligent AutomationVelocity/Execution

Distribute automation across multiple

cloud machines (set up/tear down

machines automatically) to achieve

industrial scale - run millions of process

steps several times a day.

Run only the tests necessary

through impact analysis and other

risk factors associated with a

change.

20

To Succeed

We Need to

Rethink

Automation

1 + 1 = 10x

21

Transferable Automation

1 + 1 = 4!

Automated Discovery Used to Capture a

End-to-End Business Process

Operations• Documentation

• Reuse for Robotic

process automation

QA• Documentation

• Test Processes

Application Team (IT)• Documented requirements

• Work sized, resources allocated

• Stories created for sprints

• Integration with ALM & defect tracking

Business• Visualization of process

• Detailed documentation

generated for training, IT, QA

and Ops

Reduction in manual hours

vs. test automation alone30%

22

Automated As-Is Business Process Discovery

23

Automatically Generated Documentation

QA- and Business Analyst-Friendly Screens

24

Enable Agile Delivery

• Leverage your existing quality management processes while leveraging next-generation test automation

• Use process documentation to define requirements to be associated with test processes for traceability coverage reporting

• Schedule and trigger tests directly from ALM/CD pipeline and bring back test results

25

Auto-Generated Test Cases

26

Document

Generation

Data Loading

Configuration

THINK? How Else Can We Apply Automation Across the Business for Better Results?

Configuration

Robotic Process

Automation (RPA)

Document

Generation

27

5,548 1,584 1,584No Analyze

Project/Blueprint Realization Final Prep Go Live/Hypercare

29%Reduction in

total testing

man days.

28%Increase in

focused

testing, yet

pulled in

GoLive.

30%Reduction of

2,174 total

man days for

project.

Transferable Automation Drives Additional Savings

3,960 2,186 396Analyze

9,528

7,354

75% Reduction in

Hypercare

man days.

Realization Final Prep Hypercare Total Acceleration

812

812

28

WorksoftIntelligent Automation for Enterprise/Packaged Application Projects

Process Discovery,

Documentation &

AnalyticsContinuous business insight

for digital transformation through machine learning

Business Process

Testing & RPAAccelerate Innovation,

Mitigate risk & Streamline Operations

29