the power of smart impact analysis & automated … testing...manual testing continuous testing...
TRANSCRIPT
The Power of Smart Impact Analysis & Automated Testing
Tim Rankin-Mills
Head of Automation
AGENDA
01. The goal
02. What we see
03. Continuous testing
04. Client case studies
05. A bit about TTC
The goal
01.
Smart, Continuous SAP Application Testing
SAP Application
Change
Impact Analysis
Automated Testing
Test Audit ReleaseZero
Defects
What to test?
What can be tested?What was tested?
Passed? Failed?
What we see
02.
What Impedes the Delivery of Innovative Software?
Source. Computing Research, DevOps Review 2017
Testing
49%
Manual
Approvals
23%
Planning
14%
Deployment
6%
Design
4%
Development
4%
23%26%
35% 34%37%
39% 40%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
2013 2014 2015 2016 2017 2018
(est.)
2019
(est.)
Source: World Quality Report 2017
Rising cost of Testing
What are the top 3 SAP initiatives we hear about
The 4 areas of a full S/4 HANA migration
Business Process Transformation
Adopt Agile for SAPProjects
Platform forDigital
Transformation
Move SAPWorkloads to Cloud
Continuous testing
03.
Digital transformation requires continuous testing
Digital disruption requires digital transformation
Digital transformation
requires DevOps
DevOps requires Continuous
Testing
<12% of 1950s Fortune 500
companies remain
>80% of organizations plan to
adopt DevOps
#1 technical challenge is
manual testing
Continuous Testing within Agile / DevOps
Dev Ops
“Continuous Testing runs parallel to and integrates with Agile and DevOps.”Forrester, Diego Lo Giudice,
April 14, 2017
Features of a Continuous Testing Platform
End to end Test
Automation for
SAP and other
Technologies
Comprehensive
test automation
of SAP UIs &
APIs
On-demand
Test Data
Management
for SAP
SAP Service
Virtualisation
Load Testing for
SAP Fiori and all
SAP cloud
solutions
Integration with
SAP Changes /
Impact
Analysis
SAP Business
Intelligence
Solutions
Testing
Accelerator
Packages to
speed up
implementation
Test Beyond the Boundaries of SAP
SAP WinGUI SAP BusinessClient
SAP Fiori
Test All UI & API Technologies
SAP S/4HANA
SAP WinGUI
SAP Netweaver
SAP Business Client
SAP Fiori
SAP on Mobile
HTTP(S)/ JSON/XMLoDATA RFC SOAP RFC IDOCS REST
SAP UIs
SAP APIs
Test scope reduction through LiveCompare
1,000
2,000
3,000
Used
0
Customized
Standard
Objects used
in SAP System
1,930
1,222
968
1,302
Impacted
Traditional Impact Analysis narrows the test
scope down to impacted objects.
323
-85%
156
LiveCompare reduces the test scope down to
most risky objects.
Most risky objects
(AI powered assessment)
LiveCompare and Tosca Integration
Tosca Integration
… objects covered by test cases
… objects not covered by test cases
156
323Customized
Standard
Most risky objects
(AI powered assessment)
Test gap
… test cases in Tosca
driver.FindElement(By.XPath("//table[@id='PriceList']/tbody/" + this.sumpXpath)).Text,
verificationErrors);
}
}
}
Enter Product Data
Enter Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Next
Usage
Year of Construction
License Plate Number
Enter Insurant Data
Verify Quote Details
Next
Start Date
Insurance Sum [$]
Payment Option
Price List
Insurance Sum
Gross Premium [$]
Audi
200
5
Petrol
35.000
10.000
{CLICK}
Private
{Year}
B-CD 123
{CLICK}
01/03/2016
7Mio
Yearly
1.535,22
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Verify
Select
Select
Model-BasedBrowser
Business Information
Assert.AreEqual("1.535,22",
driver.FindElement(By.XPath("//table[@id='PriceList']
/tbody/tr[10]/td[2]/b"))
.GetAttribute("value"));
CodeKeyword Driven Framework
CodeData Driven Framework
CodeRecovery Handling
CodeSynchronization Behavior
01
02
03
04
Script-BasedBrowser
Do you see the business information?
Can you scale at speed?
Model-based tests are reusable and maintainable without technical expertise
Technical InformationBusiness Information
Logical Name Value Action Logical Name Value Range
Enter Insurant Data
Create Vehicle Insurance
Enter Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Next
Usage
Year of Construction
License Plate Number
Audi
200
5
Petrol
35.000
10.000
{CLICK}
Private
{Year}
B-CD 123
Input
Input
Input
Input
Input
Input
Input
Input
Input
Input
Enter Product Data
Next
Start Date
Insurance Sum [$]
Payment Option
{CLICK}
01/03/2016
7Mio
Yearly
Input
Input
Input
Input
Verify Quote Details
Price List
Insurance Sum
Gross Premium [$] 1.535,22 Verify
Select
Select
-
-
1;2;3;4;5;6;7;8;9
Petrol;Diesel;Gas;Other
-
-
{CLICK};{RIGHTCLICK}
Private;Commercial
2015;2014;2013;2012
-
Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Next
Usage
Year of Construction
License Plate Number
Insurant Data
Quote Details
Product Data
Next {CLICK}
Start Date 01/03/2016;01/04/2016
Insurance Sum [$] 3Mio;7Mio;10Mio;15Mio
Payment Option Yearly;Quarterly;Monthly
It’s like playing Lego.
Scan Application….
Jump Start with Accelerator Packs
SAP S/4 HANA (Fiori & WinGUI)
SAP SuccessFactors
SAP Hybris C4C (Sales & Service)
SAP ECC (incl. on HANA)
SAP IS-Retail
SAP IS-Utilities
SAP IS-Manufacturing
Up to 85% coverage of SAP
test cases
Accelerate testing with prebuilt, reusable “automation templates”.
Model-Based
Test Automation
API
Testing
BI &
Big Data
Service
Virtualization
Exploratory
Testing
Risk-Based
Testing
Test
ManagementManual
TestingAnalytics
Continuous
Integration
Test Data
Management
Impact
Analysis
Load
Testing
Client Case Studies
04.
01.Complex, customized applications:
- SAP, Siebel custom
02.Highly distributed QA:
- US, LATAM, EMEA, APAC
03. Multiple failed attempts to adopt scripted test automation
04. Eliminated 85% of manual testing
Test AutomationBefore After
11% 83%
Business Risk CoverageBefore After
30% 87%
E2E ExecutionBefore After
7 weeks 4 days
Exon Mobil (US)
‘A Large FMCG Company’ (NZ)
“Frucor Suntory selected TTC, Tricentis Tosca and LiveCompare after a thorough process to select a partner and toolset to assist on our journey to Agile and Dev Ops.
Tosca and LiveCompare were undoubtedly the right tool for the job, with testing projects already providing return on investment across Frucor Suntory’s mix of SAP, Microsoft and custom iOS applications. In addition, TTC have already proven to be a valuable partner, offering flexible solutions and expertise when we have needed it.”
Customer Success
8 Years of SAP Support Packs
1M Application Changes
0 Production Defects
82% Reduction in Effort
3 Months Early Go-live
75% Saved External Resources
350% Increase in Releases
99% Reduction in Custom Testing
200+ Store Data Defects Eliminated
Global Semi-conductor Manufacturer
Global Apparel Company
Global Media Company
Consumer Products
85% Reduction in Testing
40% Increase in Release Days
0 N+1 Landscapes
A bit about
05.
Global specialists in software quality— 100% NZ Owned
— >200 consultants
— >12 offices
— >100 customers
Cincinnati
Bangkok
Auckland
Wellington
Stockholm
New York
Houston
Manila
SingaporeGlobal specialists in software quality— 100% NZ Owned
— >200 consultants
— >12 offices
— >100 customers
BelgradeGeneva
KeyServices
INTRODUCTION TO TTC
Resourcing Solutions
– Test Analyst
– Test Engineers
– Test Managers
Testing as a Service
Test Automation
(inc. Accelerators)
Environment
Management
Performance
Engineering
Test Design
Optimization
Digital
Transformation
Agile &
DevOps Dojos
Quality
Transformation
Robotic Process
Automation
Testing Fundamentals
Technology Solutions
Digital Enablement
