Final Paper/Best Practice/Tutorial Machine Learning in Test Automation

Bharathraj S – Project Manager Testing

Rajesh C – Senior Associate Testing

Banca Sella

Abstract

IntroductionWith lot of advanced tools both Licensed and Open source in fray, Test Automation is not as

successful as it is expected. What are those factors that affect Test Automation’s success?

1) The mind-set is, Manual Test cases are base for Test Automation

2) ROI not to be based on resource leverage instead should be focused on process

outcomes

3) Test Automation is designed in a way that it requires constant feedback

4) Primary focus is on AUT rather than domain and social applications/systems (focus is

Inside Out and not Outside In)

5) Confined to a tool rather than a conceptual thought process

6) Underutilization of trending technologies

7) Formulation & proportion of varied test approaches & scenarios (GUI, API/Web

Services & DB)

8) Inadequate reporting

This paper is about how to overcome the above factors and implement a successful Test

Automation process.

Traditional Test AutomationThe current approach of test automation is either to record and playback or use various

frameworks (Keyword Driven, Data Driven, Modular and Hybrid) to design and execute

predefined manual test cases. This approach is mostly Inside Out, AUT is at the centre and

other elements of test automation surrounding it.

Core Elements Core elements of Test automation are

1) Test Scenario, which is mostly derived from Use cases or Requirements followed by

Business components and Test Cases.

2) Test Data, that differentiates different Test cases & Test scenarios

3) AUT, which is mainly accessed as Objects through Object Repository

4) Driver Script, the automation scripts that binds Test Data & Test Cases together and

feeds into AUT via Objects present in Objects Repository

5) Report, the output generated by the Driver Script as a result of execution

Traditional Approach1) Test automation begins after AUT is available

2) Object repository prepared for all objects from AUT based on Test cases

3) Test Flows designed based on test cases

4) Each Test flow consists of sequence of actions to be perform on Objects from Object

repository

5) Test data prepared for every Test flow and associated

6) Test execution scheduled & executed

7) Reports analysed and defects reported manually (sometimes automated)

8) AUT changes fixed in the Test flow and re-executed

Machine Learning in Test AutomationTo address the above mentioned drawbacks and to make Test Automation a success newer

technologies & concepts such as cognitive computing, predictive analytics & machine learning

are to be adapted within Test Automation approach. Unlike traditional approach where Inside

Out process was adapted in this new approach we need to adapt to Outside In approach. What

is Outside In Approach?

1) Test automation starts much before AUT is designed or deployed

2) Manual test cases are not prerequisite for test automation anymore instead test cases are

prepared and appropriate test data are generated through cognitive computing &

machine learning engines based on domain and requirement specification

3) Social Data are processed using predictive analytics & cognitive computing engine to

generate Test data

4) Intuitive object are generated and stored in object repository using predictive analytics

& machine learning engines which are latter mapped to factual objects during runtime

5) Test Scripts are auto triggered during Build & Deployment and results are generated at

various level using a reporter engine which is mostly analytical based

6) Test failures initially are interpreted manually and the interpretation is learned by

machine learning engine which latter is automated by machine learning, predictive

analytics & cognitive computing engines

7) More mature the domain and application lesser time it takes for test automation

ConclusionAll they days we have been looking in to Test Automation at a different perspective it’s time

we change our perspective as well as our approach. With evolving cognitive computing &

machine learning we can make Test automation an intelligent machine which not only

improves resource efficiency but also the test coverage & quality of the product too.

Tools to be used1) Selenium Web Driver with Java for Object Repository creation & Driver script

2) Google / Watson / Microsoft Cognitive solutions as appropriable

3) Python scripting for developing machine learning & predictive analytics engine

4) Java & C++ will be used as appropriately inn developing various machine learning

engines

5) R will be used for developing / performing predictive analytic engine and predictive

analysis.

References & Appendix

Author Biography

Bharathraj Soundhararajan designated as Project Manager Testing practices at Banca Sella, I

head Testing & Test Automation here. Completed by MBA Operations from Symbiosis and

currently pursuing Post Graduate Diploma in Strategy Management from Indian Institute of

Management Kozhikode - IIMK & MSc Cyber Forensics & Information Security from Madras

University. I am very techno savvy and passionate person and try out all new technologies that

emerge in market. With an overall Experience of 19+ years and IT experience of more than

15+ years, designed and deployed Test Automation Frameworks for various Applications

(AUT) built using different technologies such as SAP, PEGA, JAVA and .NET. Possess

excellent communication skills and have extensive experience in interacting and working with

client across different geographic locations (i.e. US, Europe, Hong Kong, etc…). Have in-

depth knowledge on Process standards such as CMMi5 and ITIL v3. Has good track record in

team management and delivering project with stringent time lines. Had business exposure to

US & Europe geographies.

Co-Author Biography

Rajesh Chakravarthy designated as Sr Associate Testing practices at Banca Sella. An

Automation QA Engineer with over 7 years of experience in Automation Testing. Involved in

Framework Development, Automating Regression Test cases using keyword driven and Hybrid

frameworks, developing Re-Usable functions, Maintaining Shared Repository, Peer Review of

Developed Scripts by Team members. Have played key roles in Setup Test Environment &

Test Data Automation and developed Excel macro based tools and also have profound

knowledge in Automation Testing using HP-QTP and have used ALM for Test Management.

In addition to that, I have good knowledge in Mobile Testing (Android/iOS) using Appium

with Selenium webdriver. And also have good experience with web application automation

using Selenium webdriver.

THANK YOU!