sap solution manager solution documentation and test ......• scope: upgrade all sap systems...
TRANSCRIPT
Test SuiteKnowledge sharing - Solution Manager 7.2
By: Endre Rasmussen
At: Coop Norge
Date: 07.06.18
Expectations“Experience Sharing and Demonstration of Test Suite,
Solution Manager 7.2 by Coop”
Coop will share their experience of using Test Suite in Solution Manager 7.2. Learn and
understand how Solution Documentation is the foundation to successful use of Test
Suite. Gain insight and build experience in test planning, reporting, execution and defect
handling with integration to ChaRM.
Agenda
• Facts about Coop
• SolMan usage in Coop
– High level concept
• Solution Documentation
– Experience and learning points
• Test Suite
– Experience and learning points
• Demo
Facts about Coop
SAP landscape
• ECC Retail, ECC Industry, ECC slag, SCM, CRM, BW (HANA/ABAP), CAR (HANA/ABAP), SLT, Enterprise Portal, NWDI
• Three tier system landscape + sandbox
• >4000 users
• Solution Manager 7.2 SP6
Data from March 2017
28.000employees
1,5 million members
45 billionin turnover
1350stores
29,4 % marked share in
grocery
Custom Code Management
• Code Inspector (integration to ChaRM)
• CCLM – Strategic roadmap in progress
High-level concept in CoopOne Solution with integration to other modules
Business Process Improvement - PILOT
• Measure ROI based on usages statistic
in the production system
• Identify critical business processes
based on most used objects
• Identify improvement areas based on
not used objects (lack of training etc.)
• Create Solution reports for audit and
new projects
Change Control Management
• Integration to ChaRM documents
• Process structure and document
change control
Business Process Operation
• Business Process Monitoring (BPMon)
• Job Monitoring
• Data Volume Management
Test Suite
• Link to both manual and automated (CBTA)
test scripts
• Business Process Change Analyzer (BPCA)
• Scope and Effort Analyzer (SEA) - pilot
Solution Documentation
Solution DocumentationOur «Single Source of Truth»
• Solution: 1 Solution, 75 logical component groups, 2 process groups
• E2E Processes: 5 Scenarios, 9 Processes, 159 Process steps, 7 diagrams
• Modular Processes: 144 Scenarios, 458 Processes, 2395 process steps, 785
executables, 214 diagrams, 4385 test cases (only 1500 assigned to processes)
Statement: Process modelling is the foundation for control and measure of our solution, and
a common visual language between IT and Business
Solution DocumentationCoop Experience and learning points
General recommendations
- Recommend to upgrade to at least SP05 with possibility to define business process variants
- Identify the processes responsible to ensure ownership and continues maintenance
- Recommend numbering the defined processes as part of naming standard, for easier
reporting and search
- Define needed customer attributes from start to simplify the search and reporting (via
SOLMAN_SETUP)
- Use collaboration diagram in order to allow modelling of interfaces to third parties
- Define how the process can be measured and controlled (input to BPMon)
General experience
- Very good search and filtering functionality
- Experience that IE browser is slow to work with, Chrome is better but does not have the edit
online function for document handling
- User training is needed in order to navigate in a good and efficient way
Solution DocumentationCoop Experience and learning points
Relevant for Test Suite
- Decide whether to create and assign test cases on step or process level
- Create a manual test case template for consistency (expected result, actual result etc)
- Recommend to ensure test cases reflects the modelled processes, including interfaces
- Recommend to assign custom attributes to test cases, for easy test planning (example:
test phase relevance, criticality, relevant for test automation)
- Note that test cases not possible to be created for interfaces in interfaces library. This
makes it hard to report and sign off testing of critical interfaces
Modell process Create test case based on templateTest Case reflects
the process modell
Test Suite
Test SuiteExperience taken from SAP upgrade project in Coop
• Scope: Upgrade all SAP systems (3xECC, BW, CRM, SCM, Portal, TREX, Java Hub, NDWI)• Timeline: 6 months (April – October 2017)• Facts: 38 systems upgraded, 130 involved people, 1766 manual test cases executed, 159
errors detected and corrected
TEST PHASE PACKAGES TEST CASES NO RESULT
IN PROGRESS ERRORS OK DEFECTS COMPLETION (%)
SANDBOX (ST) 124 503 16 4 16 467 36 93
DEVELOPMENT (DT) 9 89 21 5 2 61 2 69
USER ACCEPTANCE TESTING (UAT)
251 931 0 0 0 931 119 100
PRODUCTION VERIFICATION (PRODT)
65 308 0 0 1 307 2 100
TOTAL OF ALL TEST PHASES 449 1831 37 9 19 1766 159 100
One of the key success factors was the preparation work, identify and secure test coverage of critical business processes and related interfaces
Test SuiteHigh level Test flow
• Run BPCA to scope test
• Creation of Test Plan, Test
Package and Test Case and
Tester assignment
• Creation of Test Reports
• Test defect handling in ITSM
• Using Defect Correction in
ChaRM (close link to test defect)
• Test Execution via Tester
Worklist
• Test Note creation
• Test Defect creation
Tester DeveloperTest
Manager
Planning Execute Defect handling
Test Suite - Test PlanningExperience and Learning points
Experience
• Nice integration with Solution Documentation, structure is maintained and well known
• Note that changes in test case (SolDoc) will not be visible in existing test packages, new test
package needs to be created and assigned to tester
• Missing mass update function of tester in test packages for Test Manager, comes with SP05
• Good reporting possibilities (drill down in all levels)
• BPCA analyse is complex and requires detailed investigation in order to understand the result
• No experience in Test Case sequence, possible but not used
Learning points
• Decide on which level in structure the test plan and test packages should be created for easy
reporting and follow up
Test
Manager
Example of report:
Test Suite - Test ExecutionExperience and Learning points
Experience
• Tester recognize the structure in tester worklist from the process structure, which is good
• Too many pop ups/different windows when working with manual execution. Solved with Focus Build
• Test Cases opens in a separate window, not embedded as in 7.1. Solved with Focus Build
• Many refresh buttons under Tester Worklist. Needs to be refreshed manually
• Start Test Execution button makes sense if you have test case on process step level (Coops
decision to have test case on process level). Solved with Focus Build
• No experience in TBOM recording, decided not to use refer to statement above
Learning points
• Decide the use of Test Note and Test Result, and method (check in check out or upload new
version)
• Evaluate the available Focused Build solutions offered by SAP related to Test Suite (Simplified test
execution etc)
NEWS: Focused Build for 7.2 - New manual test solution with grid approach fully
embedded in Test Suite etc – link (slide 27-45)
Tester
Test Suite - Defect HandlingExperience and Learning points
Experience
• Nice follow up document integration between Test Defect (Incident) and Defect Correction
(ChaRM), text and field entries can be copied over
• (SP05) No interlink when processing Test Defect and Defect Correction. Developer needed to
work on both Test Defect and Defect Correction document when updating status etc
• No direct link to test notes from Test Defect, only link to test case in SOLDOC (where actual
result is blank) Our method was to attach test note to Test Defect
• Note that Testers must go into ITSM in order to reply and confirm reported Test Defects (direct
link from tester worklist or mail notification)
Learning points
• Analyse the standard Defect Handling workflow and evaluate if customizing is needed
• Upgrade to SP06 in order to get the status interlinked between Test Defect and Defect
Correction which can cause manual work and continual progress*
*Shipped with SP06: Status profiles for Test Defect and Defect Correction interlinked – link (slide 24)
Developer
Demo
SolMan 7.2 SP6