How can test automation validate the end-to-end business

process to help ensure 24/7 operational continuity

– A customer case description with Metso Corporation.

Krista Männistö, MetsoRich Lock, Worksoft

Oct 21th, 2014

© Metso

Metso as a corporation –Products and usage of SAP Metso is a leading process performance provider with customers in the mining, construction, and oil & gas industries.

- We employ around 16,000 professionals in 50 countries.- SAP is used as ERP currently in 25 different countries, mainly in Mining and

Construction sites. - Metso has monthly releases and main releases quarterly. SAP was

upgraded to EhP7 level 05/2014.

Krista Männistö

© Metso

Agenda Purpose of this presentation is to show how we in Metso have automated our business processes and what business benefits have been gotten via automated testing inside Global Applications team.

- Metso as a corporation – Products and usage of SAP

- Test automation project at Metso – From beginning to present

- Current level of test automation – Amount of test scripts

- Manual testing versus test automation – Regression test analysis

- Benefits of test automation – Key learnings of the project

- Benefits of test automation – Possible impact on main release schedules

- Test automation use cases – Impact on existing processes

- Solution Manager and Worksoft Certify integration – Process description

- Next steps with test automation – Project targets for year 2014

I will be available for more questions and live demos from the system on the Worksoft/Bilot booth between 14.00 – 15.00.

Krista Männistö

© Metso

Test automation project at Metso –From beginning to presentTest automation project was launched at Metso in February 2014. Target of the project is to automate 80-90% of manual SAP related testing during year 2014. At the moment Metso has documented and automated 45 end-to-end processes and more and more are being finished all the time.

Krista Männistö

© Metso

Current level of test automation –Amount of test scripts

• Currently Metso has automated 45 E2E test scripts.These E2E scenarios include:

- 182 Transactional processes- 3981 Process steps

• Also some utilities and mass upload/data creation scripts have beencreated in Worksoft Certify side.

Krista Männistö

© Metso

Manual testing versus test automation –Regression test analysis• Usually only about 40 E2E test scenarios from existing manual test

scripts are tested during regression testing cycle with 80-90 differentvariants for separate CoCodes.

• Importance of master data quality during testing!

Krista Männistö

• Test automation was analyzedagainst manual test result fromlast regression test cycle inAugust for SD module.

- Cost savings could be achievedfrom hours saved in testing and quality of testing.

- Manual testing for 28 variantstook 90 hours and test automation only 3 hours. Test automation 97% faster

© Metso

Test automation use cases –Impact on existing processes

• Worksoft Certify test automation tool could be used for many differentpurposes. Possible use cases could be for example:

- Unit, integration and regression testing- Testing of outputs- Test documention for individual change requests for business users- Mass uploads either to development, test or production clients- Actual configuration steps- Creation of master data- Process documentation and modelling

• Successfully implementing a new test management method intoorganization transforms existing processes – previously manuallycoordinated manual testing process with emails and excels isreplaced with Solution Manager Test Management where automatedtests can be executed.

Krista Männistö

© Metso

Solution Manager and Worksoft Certify integration – Process description

Tests are monitored and performed via SAP Solution manager which launches external tool called Worksoft Certify. Certify then launches the actual test script and tests are run for example in SAP test client. Results and data are transferred back to SAP Solution Manager as logs.

Krista Männistö

© Metso

Benefits of test automation –Key learnings of the project

Test automation will make internal processes more flexible and improves productivity by releasing time and resources for other more productive work than manual testing and testing could be performed more frequently and whenever needed. This creates more agility to respond to business requirements and increases our competitiveness.

Key learning of the project has been to understand how important testing truly is in order to deliver changes to production environment in a safely manner. We have also only now came to realize how unreliable and unrepeatable manual testing results have earlier been. This has caused a huge risk in our way of operating with change management and also delayed the end to end lead time of change requests.

Krista Männistö

© Metso

Next steps with test automation –Project targets for year 2014

• Target is to automate 80-90% of the E2E test scenarios thatcurrently are tested manually by end of year 2014.

- Current test scripts per module are also checked and modified and new onesare created if needed.

• Start test scheduling with Solution Manager (lights-out testingduring nights).

• Start using test automation instead of manual testing for differenttest cycles. Target is to use automated testing instead of manualtesting in next main release during November for SD, MM, WM,PP, FI, CO and CS modules. This means also that testing effortswould be done by own internal ERP team and not by offshoretesting team.

- This would save roughly 300 manual testing hours from next regressiontesting cycle.

Krista Männistö