powerpoint slides
TRANSCRIPT
Mar 20, 2008
www.arsin.com
Packaged Test Automation for SAP
Danis Yadegar (President & CEO
Presenter
2CONFIDENTIAL
© 2008 Arsin Corporation. All Rights Reserved
Agenda
SAP Vs Custom Applications
Testing needs of SAP Applications
Summary
Current Approach – Challenges & Limitations
Solution
Q&A
3CONFIDENTIAL
© 2006 Arsin Corporation. All Rights ReservedCONFIDENTIAL
© 2006 Arsin Corporation. All Rights ReservedCONFIDENTIAL
© 2008 Arsin Corporation. All Rights Reserved
SAP Vs Custom-built Applications
4CONFIDENTIAL
© 2006 Arsin Corporation. All Rights Reserved
SAP Vs Custom Application Testing
CONFIDENTIAL© 2008 Arsin Corporation. All Rights Reserved
• Transaction Focused Testing
• Configurable and Customizable Execution
Components
• Configurable and Customizable Business Rule
Validation
• Complex and Large set of Test Data
• Data in QA Environment is Hard to Refresh
SAP
• GUI/API Focused Testing
• Reusable GUI/API Execution Components
• Screen based Edit Rules & Referential Integrity
checks
• Limited and Controlled set of Test Data
• Easy and Full Control over QA Environment,
Easy to reset baseline test data
Custom
5
Change Drivers
Within CompanySAP
Upgrades
New Modules Additions
Support Packs
Business Unit Changes
Mergers and Acquisitions
Partner & Customer Integration
Business Process Changes
Enhancements/Fixes
System Environment
Hardware and OS Changes
Consolidations
Recovery From Crash
Disaster Recovery Readiness
Data Base System Changes
Other Applications
SAP New Dimension Products
Packaged Applications
Custom Applications
Complimentary Software
Integration Platforms
Regulatory Compliance
Sarbanes-Oxley
FDA System Validation
InternationalRegulatoryComplianceRequirements
6CONFIDENTIAL
© 2006 Arsin Corporation. All Rights ReservedCONFIDENTIAL
© 2008 Arsin Corporation. All Rights Reserved
Summary
Testing SAP requires a different set of tools, in addition to the commercially available GUI testing tools Testing SAP requires a new testing methodology and process Testing SAP requires people with different knowledge areas – business process, SAP configuration & development
7
Current Approach - Limitations
SAP Test Automation Manual and screen based validations
Input and expected data is managed in spread sheets, which is difficult to maintain
Managing changes – Inability to rapidly change test artifacts during changes in the systems
8CONFIDENTIAL
© 2008 Arsin Corporation. All Rights Reserved
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Test Automation Challenges - An Example
LEGENDExecution Validation
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Order To Cash Scenario
Goods Issue
Delivery Stock
Overview Display
Condition Display
Customer
Display Sales Order
SalesOrder
Billing Cust. Acct.
BalanceDisplay
Display Acct.
Document
Balance Display
Stock Overview
Display Material
Document
DisplayOut BoundDelivery
Cust Acct. Balance Display
DisplayAcct.
Document
Incoming Payment
Cust. Acct. Balance Display
Balance Display
Display Acct.
Document
DisplaySalesOrder
9
Incoming Payment
Billing Goods Issue
Problem One: Data Management
Order To Cash
DeliverySalesOrder
Sales Order Test Script
Stock Overview
VA01 VA03 XD03 VK13 MMBE
Execution Validation
Display Condition
Display Customer
Display Sales Order
SalesOrder
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
10
Problem Two: Validation
1:3 Ratio – Execution to Validation
Screen Based Access - No Access to Internals
High Degree of Test Automation Skill Required
3X as Hard to Maintain
The Only Way to Reduce the Functionals’ Workload
Sales Order Test Script
Stock Overview
VA01 VA03 XD03 VK13 MMBE
Execution Validation
Display Condition
Display Customer
Display Sales Order
SalesOrder
11
Problem Three: Managing Change
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Order To Cash Scenario 1
Goods Issue
Delivery Stock
Overview Display
Condition Display
Customer
Display Sales Order
SalesOrder
Billing Cust. Acct.
BalanceDisplay
Display Acct.
Document
Balance Display
Stock Overview
Display Material
Document
DisplayOut BoundDelivery
Cust Acct. Balance Display
DisplayAcct.
Document
Incoming Payment
Cust. Acct. Balance Display
Balance Display
Display Acct.
Document
DisplaySalesOrder
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Order To Cash Scenario 1
Goods Issue
Delivery Stock
Overview Display
Condition Display
Customer
Display Sales Order
SalesOrder
Billing Cust. Acct.
BalanceDisplay
Display Acct.
Document
Balance Display
Stock Overview
Display Material
Document
DisplayOut BoundDelivery
Cust Acct. Balance Display
DisplayAcct.
Document
Incoming Payment
Cust. Acct. Balance Display
Balance Display
Display Acct.
Document
DisplaySalesOrder
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Order To Cash Scenario 1
Goods Issue
Delivery Stock
Overview Display
Condition Display
Customer
Display Sales Order
SalesOrder
Billing Cust. Acct.
BalanceDisplay
Display Acct.
Document
Balance Display
Stock Overview
Display Material
Document
DisplayOut BoundDelivery
Cust Acct. Balance Display
DisplayAcct.
Document
Incoming Payment
Cust. Acct. Balance Display
Balance Display
Display Acct.
Document
DisplaySalesOrder
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Order To Cash Scenario 1
Goods Issue
Delivery Stock
Overview Display
Condition Display
Customer
Display Sales Order
SalesOrder
Billing Cust. Acct.
BalanceDisplay
Display Acct.
Document
Balance Display
Stock Overview
Display Material
Document
DisplayOut BoundDelivery
Cust Acct. Balance Display
DisplayAcct.
Document
Incoming Payment
Cust. Acct. Balance Display
Balance Display
Display Acct.
Document
DisplaySalesOrder
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Order To Cash Scenario 1
Goods Issue
Delivery Stock
Overview Display
Condition Display
Customer
Display Sales Order
SalesOrder
Billing Cust. Acct.
BalanceDisplay
Display Acct.
Document
Balance Display
Stock Overview
Display Material
Document
DisplayOut BoundDelivery
Cust Acct. Balance Display
DisplayAcct.
Document
Incoming Payment
Cust. Acct. Balance Display
Balance Display
Display Acct.
Document
DisplaySalesOrder
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Order To Cash Scenario 1
Goods Issue
Delivery Stock
Overview Display
Condition Display
Customer
Display Sales Order
SalesOrder
Billing Cust. Acct.
BalanceDisplay
Display Acct.
Document
Balance Display
Stock Overview
Display Material
Document
DisplayOut BoundDelivery
Cust Acct. Balance Display
DisplayAcct.
Document
Incoming Payment
Cust. Acct. Balance Display
Balance Display
Display Acct.
Document
DisplaySalesOrder
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Validation Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Spreadsheet
Execution Data
Order To Cash Scenario 1
Goods Issue
Delivery Stock
Overview Display
Condition Display
Customer
Display Sales Order
SalesOrder
Billing Cust. Acct.
BalanceDisplay
Display Acct.
Document
Balance Display
Stock Overview
Display Material
Document
DisplayOut BoundDelivery
Cust Acct. Balance Display
DisplayAcct.
Document
Incoming Payment
Cust. Acct. Balance Display
Balance Display
Display Acct.
Document
DisplaySalesOrder
New billingfield
12
Current Approach - Limitations
Knowledge Management SAP testing is spearheaded by functional experts who map the Business
Process to SAP modules During the initial implementations there is more focus on testing than on
capturing knowledge residing with functional & technical consultants As a result, the regression library is incomplete for future support packs
and upgrades
Methodology & Process Formal Testing is addressed as one of the last activities after completion
of development and unit testing ASAP methodology guidelines for testing are very high level and does not
focus on detailed activities to be performed by the QA team during various phases implementation
As a Result testing gets pushed till the end and quality is compromised due to lack of early engagement and preparation
13CONFIDENTIAL
© 2008 Arsin Corporation. All Rights Reserved
Solution
Methodology & Process
Knowledge Management
Technology
14CONFIDENTIAL
© 2008 Arsin Corporation. All Rights Reserved
Technology Solution
Flexible multi-platform GUI test automation tool – RFT, QTP, TP
Scoping Workbench
Test Automation Framework for SAP Landscape
Configurable and Customizable Test Components
Built-in Data and Business Rule Validations for:
Standard and Custom Transactions
Custom Processing Programs
Inbound/Outbound Interfaces
Sophisticated Test Data Management
Pre and Post Change Comparators for Documents and Tables
15
Test Automation Process Flow
System Under Test
Effecta™ Validation Engine
SAPERP
Core Business Processes
SAP GUI
Database
Interfaces
QA Mapper™
Test RequirementsTest Cases
Test ProceduresMeta Data of ECsDefinition of VC
Data Sets
Wrapper Extracts
Test Case Info
EC Meta DataVC Name
Data Set InfoFrom
QA Mapper DB
GUIAutomation
Tool
EC Scriptson
Shared Drive
Wrapperon the
Shared Drive
Runs ECExtracts Keys
Pass it to Effecta forValidation
2
1
3
4
5
Validation Reports
6
RFT Log
7
16CONFIDENTIAL
© 2008 Arsin Corporation. All Rights Reserved
Test Automation Solution for Interfaces
QA Mapper/GUI Tool
17CONFIDENTIAL
© 2008 Arsin Corporation. All Rights Reserved
Methodology & Process Needs
Compatible with SAP implementation Lifecycle
Process to Test
Configuration & Modifications
Conversions
Interfaces
Connectivity
Security
Archiving
Production Fixes and Enhancements
Support Packs & Upgrades
Include Processes to handle compliance requirements
18CONFIDENTIAL
© 2008 Arsin Corporation. All Rights Reserved
SAP QA Process
QA Activities - SDLC Phase Mapping
Define Design Build / Test Prod. Readiness Delivery Warr. Support
QA
Ea
rly
En
ga
ge
me
nt
Ass
ign
QA
re
sou
rce
for
Ear
ly E
nga
gem
en
tR
evi
ew
BR
, H
igh
Le
vel S
izin
g, T
ime
line
Page: 1 / 1Revision date: May 16, 2007
File: VisioDocument
Regression Library UpdatesPM or Designee to review
CMs and change records approved
QA Certification Document/
Risk assessment document
-business sign-off-project team
review
Review High Level Sizing
Inputs:Project Charter
SCOPE Document
Project PlanInitial
Requirements
Outputs:High Level
Resource and Time Estimates
Prioritize QA test requirements
-business SME & designer sign-offQ
A le
ad
co
mp
lete
s p
ha
se
ch
ecl
ist
Test Execution
QA Environment Readiness Review
QA Test PlanningInputs:
“To Be” System Design
“To Be” Business Process Flows
and BPPsFunctional Specs
Outputs:High Level QA
Test Plan
Review Documents
Technical SpecsDevelopment Test
PlanChange Records/Development list
Develop detailed test plan,
test requirements, test cases, test
procedures
Complete detailed sizing
Data Mining
Connectivity Testing
Identify / Create baseline data
Turnover to QA Production
Support
QA Kick-off meeting
Project Plan
QA Pre-Certification
Meeting
QA Metric – Open Defects
Captured
Pre-Production Readiness
Audit
Post Prod Readiness Audit
Process MetricsSchedule Variance
Effort VarianceCost Variance
Root Cause Analysis
Quality MetricsDefect Detection
Effectiveness
Final Audit
Test Emergency Requests
QA
lea
d c
om
ple
tes
ph
ase
c
he
clis
t
QA
lea
d c
om
ple
tes
ph
ase
c
he
clis
t
Test Automation Updates
QA Resource PlanningInputs:
High Level QA Test PlanOutputs:
QA/Partner Resources
Requirements/Assignments
Lessons Learned
Conduct Survey
QA
lea
d c
om
ple
tes
ph
ase
c
he
clis
t
Test Automation Requirements
Test Post Go Live Changes
Knowledge Transfer
Conversions and Master Data
Setup
19CONFIDENTIAL
© 2008 Arsin Corporation. All Rights Reserved
SAP QA Knowledge Management
QA Process Retains Knowledge through
Detailed Documentation of regression library including validation rules
Automated Execution
Automated Validation of Business Rules
Training Program
SAP Professionals on QA Processes
Functional & Technical Training for Non-ERP IT Professionals
Test Automation Frameworks
20
Ideal Testing Landscape
Basis Administrator
ABAPDeveloper
Application XDeveloper
SAP FunctionalAnalyst
Business ProcessAnalyst
ComplianceAuditor
TEST REQUIREMENTS TEST CASES TEST PROCEDURETEST EXECUTION
&METRICS & DEFECTS
Test Management
Test Component Library
SAP FI MM SD
PP PS FS
SRM CRM APO
Interfaces
EDI
Batch
EAI
Non-SAP Applications
ClientServer AS400 Web Other
Business Process Test Library
Sell-Side Processes
Buy Side Processes
Manufacturing Processes
Financial Processes
GUI Test Automation Tool
EffectaTM SAP Validation Engine
PROVIDEBUILDINGBLOCKS
BUILDMAINTAIN
BUILDMAINTAIN
QA AutomationEngineer
QA Analyst
EXECUTE
SME
QA Analyst
21
IBM-Arsin’s Test Automation Solution
Execution Engine
Rational Functional Tester
System Under Test
Effecta™ Validation Engine
SAPERP
Core Business Processes
SAP GUI
Test Scripts Database
Interfaces
Web Browser
Web
Test Req
Test Cases
Test Procedures
Test Data
QA Mapper™
Database
23CONFIDENTIAL
© 2006 Arsin Corporation. All Rights Reserved
Q&A