1

Web Application testing with SAHI tool

Shalu (M.Tech) Mrs. Neha Garg (Asst. Professor)

Department of Computer Science and Engineering Department of Computer Science and Engineering

Graphic Era University Graphic Era University Dehradun, India Dehradun, India

ABSTRACT

Sahi is a tester friendly tool. There are many commercial tools and open source frameworks for

automated testing of web applications. Sahi is a tool for automating web application testing. Sahi

is an open source tool that allows recording and replaying across browsers. It provides different

language drivers for writing test scripts, and supports dynamic web applications. It abstracts out

most difficulties that testers face while automating web applications. Some salient features

include excellent recorder, Platform and browser independence, No XPaths, No waits,

Multithreaded (parallel) playback, Excellent Java interaction, Inbuilt reporting. Some of the

Advantages of sahi are:

Sahi is an engineering solution for an engineering problem.

Sahi also exposes the ability to be exact when needed, making Sahi very suitable for any web

application.

Browser Platform Independence

Sahi, because of its architecture, supports any browser with javascript support and any Operating

System with Java support

Smart Identification Mechanisms

The Javascript DOM exposes a lot of properties of various elements on the browser. Sahi uses

these different properties to identify elements. One significant point is that Sahi gives the visible

text a higher priority than other code related properties like css style. This helps in two ways

� Writing and maintaining test scripts become very easy. What you see is what you script.

� It validates what a user sees and not what a developer coded.

Sahi automatically goes through frames, iframes etc. to find elements. One does not need to

explicitly select a frame in the automation script. So if developers moved the login fields into an

iframe, Sahi will still work with your older scripts.

KEYWORDS: web application, java scripts, DOM, multithread.

2

Table of Contents Page No.

1. Introduction 6

1.1 Features 6

1.2 Limitations 6

2. Technique description 8

2.1 Sahi Architecture 8

2.2 Sahi Advantages 9

3. Description of the tool used for simulation 10

3.1 Getting Started 10

3.2 Prerequisites 10

3.3 Download Sahi OS 10

3.4 Installation of Sahi 10

3.5 Sahi starting 11

3.6 Sahi Scripting Basics 11

4. Experiments and Results 14

4.1 Recording through Sahi 14

4.2 Playing back through Sahi 16

4.3 View Logs 16

4.4 Sahi Excel Framework 16

4.5 Play back reports/logs 18

5. Problem extended for dissertation 19

6. References 20

7. Figures

Fig -1 8

Fig -2 10

Fig -3 10

Fig -4 11

Fig -5 14

Fig -6 14

Fig -7 15

3

Chapter 1

1. Introduction

Sahi is an automation tool to test web applications. Sahi injects javascript into web pages using a

proxy and the javascript helps automate web applications. Sahi is a tester friendly tool. It

abstracts out most difficulties that testers face while automating web applications. Some salient

features include:

� Excellent recorder

� Platform and browser independence

� No XPaths

� No waits

� Multithreaded (parallel) playback

� Excellent Java interaction

� Inbuilt reporting

1.1 Features

� Browser and Operating System independent

� Powerful recorder which works across browsers

� Powerful Object Spy

� Intuitive and simple APIs

� Javascript based scripts for good programming control

� Version Controllable text-based scripts

� In-built reports

� In-built multi-threaded or parallel playback of tests

� Tests do not need the browser window to be in focus

� Command line and ant support for integration into build processes

� Supports external proxy, HTTPS, 401 & NTLM authentications

� Supports browser popups and modal dialogs

� Supports AJAX and highly dynamic web applications

� Scripts very robust

� Works on applications with random auto-generated ids

� Very lightweight and scalable

� Supports data-driven testing. Can connect to database, Excel or CSV file.

� Ability to invoke any Java library from scripts

1.2 Limitations

• Framesets/pages with frames/iframes loading pages from multiple domains is not

supported. Sahi cannot handle pages which have other pages from different domains

embedded in them using iframes or frames. So you cannot have a page from google.com

4

having an iframe with a page from yahoo.com. Note that this is not the same as switching

between domains, where you navigate from a google.com page to a yahoo.com page,

which will work in Sahi.

• File upload field will not be populated on browsers for javascript verification. File upload

itself works fine

The very basic difference between Sahi and other traditional tools (selenium, QTP, etc.) is the

parallel execution. Test execution for unattended runs (Built in Ant support for integration with

build process or batch runs) which saves huge time, makes overnight regression test execution

possible. Ant support offers a lot of freedom to automate various scenarios which were not that

easy earlier. e.g do something in app , run a batch process on the server then go back to app and

check something. User can write atomic tests independent of each other for parallel execution

which means no need to maintain start and end state of each test.

With intelligent object spy and some really cool in built functions it is pretty easy to identify

each element uniquely. Sahi being based on JavaScript allows rapid script development /

maintain with a very short learning curve.

Sahi has Robust Exception-handling (continue with alternate flow on failure) as well.

Sahi's Smart Reporting allows user to view report in HTML as well as Junit format.

Sahi's OS and Browser independence gives lot of power and freedom to tester (no rework for

different OS or browser).

Simple debugging and logging helps tester to write / maintain scripts easily and of course it is

version control friendly – zero binary files. Sahi being dynamic enough we can build a

framework around that to have a data driven capability (Input Data, Output Data, Meta Data) or

even further a framework where everything is abstract (Business Logic, Sahi scripting , data,

element identification). In other words Sahi not only has most of the stuff that an automation tool

should have it is also smart and easy for a rapid script development and minimum maintenance.

5

Chapter 2

2. Technique description

2.1 Sahi Architecture:

The architecture of Sahi allows Sahi to be used on any browser or operating system. Sahi relies

on two core technologies/concepts:

1. HTTP proxy – to inject code

2. Javascript code – to find elements and emulate actions

Both these technologies are basic building blocks of internet technologies and will necessarily be

supported by all browsers, making Sahi very easily extensible to newer browsers or newer

versions of browsers.

Sahi uses a HTTP proxy at its core to inject javascript into web pages. The injected Javascript

uses custom code to identify elements on the browser and simulate actions like click, type etc. on

them.

Fig-1: Java based proxy server injects java scripts code

Html responses which pass through the proxy are modified such that JavaScript is injected at the

start and the end of the response. This allows the browser to record and playback scripts and talk

back to the proxy when needed. Apart from handling requests for pages that the browser

requests, Sahi’s proxy also handles custom commands related to recording, playback etc. which

the browser sends.

6

2.2 Sahi Advantages:

Sahi is an engineering solution for an engineering problem. Sahi also exposes the ability to be

exact when needed, making Sahi very suitable for any web application.

i) Browser Platform Independence:

Sahi, because of its architecture, supports any browser with javascript support and any Operating

System with Java support

ii) Smart Identification Mechanisms:

The Javascript DOM exposes a lot of properties of various elements on the browser. Sahi uses

these different properties to identify elements. One significant point is that Sahi gives the visible

text a higherpriority than other code related properties like css style. This helps in two ways

• Writing and maintaining test scripts become very easy. What you see is what you script.

• It validates what a user sees and not what a developer coded.

Sahi automatically goes through frames, iframes etc. to find elements. One does not need to

explicitly select a frame in the automation script. So if developers moved the login fields into an

iframe, Sahi will still work with your older scripts.

iii) UI Relations

Sahi is slightly different from other automation technologies. It relies less on the underlying code

and more on the business intention of the user interface. Code changes are frequent in an

evolving application and relying on business intention is a more stable way of writing

automation test cases. Sahi does this through what is called UI Relations. UI Relations help

identify one element with respect to another element which communicates strong business

intention.

iv) Parallel Playback

Sahi allows Parallel or Multithreaded playback. Multiple tests can be run simultaneously to

reduce playback time, thus reducing your build time and shortening your feedback cycle. One

can also spread the tests across machines or on the cloud and get one consolidated report, all

stored in a database.

7

Chapter 3

3. Description of the tool used for simulation

3.1 Getting Started

3.2 Prerequisites

Java 1.5 or above is needed for running Sahi.

3.3 Download Sahi OS

Download Sahi OS from http://sahi.co.in

3.4 Installation of Sahi

Once Sahi is downloaded, double click on the jar file to run the installer.

On starting the installer, follow the steps below:

Fig-2: Installing Sahi

Fig-3: Sahi installed

8

3.5 Starting Sahi

Start Sahi Dashboard by any of the following methods:

1) Double click on the desktop shortcut

2) Go to Start -> All Programs -> Sahi -> Start Sahi

3) Start from the command line.

Windows: – Go to <Sahi>\userdata\bin and run start_dashboard.bat

The Sahi Dashboard starts the Sahi proxy, and allows launching of different browsers. Sahi

automatically modifies the browser’s proxy settings, so that requests go through the Sahi Proxy

(localhost:9999)

Fig-4: Sahi dashboard

3.6 Sahi Scripting Basics

Sahi script is based on javascript, but they are not the same. Sahi script is parsed by the proxy and the parsed script is fully valid javascript which is executed by the rhino javascript engine. Below are the normal constructs used in Sahi script. They are the same as javascript except for the mandatory $ used in variables. Statements:

Statements are normal lines of code. They end with a semi-colon. For example, _click(_link("Login"));

9

Variable declaration:

Syntax var $variableName = value;or var $variableName; // declaration $variableName = value; // assignment

Example var $username = "SahiTestUser";

var $password; // declaration; $password = $username + "_password"; // "SahiTestUser_password" Note All variables start with a $. The keyword var is used for local variables.

Function declaration:

Syntax function functionName($parameter1, $parameter2) { // statements }

Example //function declaration function login($usr, $pwd){ _click(_link("Login")); _setValue(_textbox("username"), $usr); _setValue(_password("password"), $pwd); _click(_submit("Login")); } //function call login("sahi_user", "secret");

if statements:

Syntax if (condition) { // statements }

Example if ($username == "PartnerUser"){ _click(_link("Partner Login")); } If the if condition is page dependent, use if (_condition(booleanExpression)) For example, if (document.loginForm.userName =="karthik"){...} should be written as if (_condition(document.loginForm.userName == "karthik")){...}

NOTE _condition need not be added if the condition does not depend on any page’s DOM. for loops:

Syntax for (var $i=0; $i<$max; $i++){ // statements }

Example // This loop will login with user1, password1, user2, password2 etc. // login and logout are custom functions. for (var $i=0; $i<10; $i++){ login("user"+$i, "password"+$i);

10

logout(); }

Note Do not use a condition which depends on a page’s DOM. for (var $i=0; $i<document.links.length; $i++) {...} //WRONG If you want to iterate over page DOM variable, first take a snapshot of it using _set, and then use the variable. For example, <browser> var $anchors = []; function getLinkIds(){ var retVal = []; var links = document.links; for (var i=0; i<links.length; i++){ retVal[i] = links[i].innerHTML; } return retVal; } </browser> _set ($anchors, getLinkIds()); for (var $i=0; $i<$anchors.length; $i++) {...} //RIGHT

11

Chapter 4

4. Experiments and Results

4.1 Recording through Sahi

� Click on any browser on the Dashboard. A browser window should open with the following screen

Fig-5: Sahi browser window Press ALT and double click on the window which you want to record. The Sahi Controller will pop up. (If that does not work, press CTRL and ALT keys together and then double click. Make sure popup blockers are turned off)

Fig-6: Sahi Controller

12

� On the controller, go to the Record tab. � Give a name for the script, and click ‘Record’. (.sah is optional)

Fig-7: Sahi record

� Navigate on your website like you normally would. Most actions on the page will now get

recorded. i) The currently recorded step is visible in the “Evaluate Expression” box

ii) All recorded steps are visible in the “Recorded Steps” box

� Add an assertion:

i) Move the mouse over any html element while pressing Ctrl key. The Accessor field will

get populated in the controller.

Fig-8: Sahi browsing ii) Click the “Assert” button to generate assertions for the element. They will appear in the

“Evaluate Expression” box.

iii) Click “Test —>” to check that the assertions are true. You can evaluate any javascript

using “Evaluate Expression” and “Test —>”. Actions performed via the controller will

not be automatically recorded. Only actions performed directly on the page are

automatically recorded. This lets you experiment on the webpage at recording time

without impacting the script.

13

iv) Once satisfied, click on “Append to Script”. This will add the assertions to the script. v) Click “Stop” to finish recording.

Note that the controller can be closed and reopened at any time, without disrupting recording.

The recorded script is stored in <sahi_pro>\userdata\scripts directory. The recorded script can be

viewed and edited easily through any text editor. Sahi Scripts are simple text files which use

Javascript syntax.

The script can be edited even while recording, so that logical segregation into functions etc. can

be done as recording happens.

4.2 Playing back through Sahi

Running a test from the controller

� Open the Sahi controller (ALT-Dbl click on the page).

� Click on “Playback” tab

� Enter the script name in the “File:” field (with the help of the auto completion feature)

� Enter the start URL of the test. Eg. If you had started recording from

http://sahi.co.in/demo/training/, use that URL.

� Click ‘Set’.

� Wait for the page to reload.

� Click ‘Play’.

Steps will start executing, and the controller will be updated accordingly. Once finished,

SUCCESS or FAILURE will be displayed at the end of the steps.

4.3 View Logs

On the controller, go to Playback tab and click on “View Logs” link at the bottom right. It will

open a window with the results neatly formatted in HTML.

Clicking on a line in the logs will drill down to exact line in script. Logs show all the assertion in

green. If the assertion has failed it will show in red. You can click on any of these lines to go into

the line of script to debug.

4.4 Sahi Excel Framework

Sahi introduces an advanced way of automation via its Excel Framework. The Sahi Excel

Framework allows testers to write their test cases in Excel and run it from Sahi. Often a testing

team consists of a mix of subject matter experts, some manual testers and testers with some

automation experience. Writing tests in the language of the business allows all stake holders to

participate and derive value out of the automation process.

14

Sample Excel Sheet For example, a simple test written as an Excel sheet looks like this:

TestCase Key Word Argument1 Argument2 Argument3

loadSahi books_lib.sah

Check shopping cart

total [Documentation] Smoke test for add books

Login "test" "secret"

addBooks 3 2 1

verifyTotal 1640

Logout

Test login error message [Documentation] Checks Invalid login message

login "test"

"bad

password"

verifyNotLoggedIn

verifyErrorMessage "Invalid username or password"

These tests talk mostly in the language of the business (also called a Domain Specific Language or DSL for that business), and hideaway all the implementation details of clicking buttons and populating textboxes. Loading Sahi script libraries

The implementation details are moved into an included Sahi script, which is linked to this Excel sheet via the initial statement.

loadSahi "books_lib.sah"

The code in books_lib.sah is given below:

function login($username, $password){ _setValue(_textbox("user"), $username); _setValue(_password("password"), $password); _click(_submit("Login")); } function addBooks($numJava, $numRuby, $numPython){ _setValue(_textbox("q", _near(_cell("Core Java"))), $numJava); _setValue(_textbox("q", _near(_cell("Ruby for Rails"))), $numRuby); _setValue(_textbox("q", _near(_cell("Python Cookbook"))), $numPython); _click(_button("Add")); } function verifyTotal($total){ _assertEqual($total, _textbox("total").value);

15

} function logout(){ _click(_button("Logout")); } function verifyNotLoggedIn(){ _assertExists(_textbox("user")); } function verifyErrorMessage($msg){ _assert(_isVisible(_div("errorMessage"))); _assertEqual($msg, _getText(_div("errorMessage"))); } Executing the Excel Sheet

Executing the Excel sheet is no different from executing a Sahi script.

Just copy the Excel sheet into sahi/userdata/scripts directory (or inside a sub-directory) and the

xls file will be visible in the drop down in Controller Playback tab.

Choose the file, Click "Set" and then "Play" to playback.

4.5 Play back reports/logs

On execution, Sahi generates logs showing success or failure. Logs are visible from the "Logs"

link in Playback tab.

IMPORTANT NOTES

i. You may need to close the Excel sheet before running it ii. Only the first sheet will be executed. The first sheet should be called Sheet1 and should not

be renamed iii. MS Excel needs to be installed on the system iv. Due to a bug, numbers need to be formatted as text in Excel. One can do this by entering a

single quote before entering the number For example type '23 instead of 23.

16

Chapter 5

5. Problem extended for dissertation

Large organizations or organizations such as banks and financial institutes, telecommunication

operators, hospital and health institutes and public or governmental bodies have many reasons

for addressing information security very seriously. Legal and regulatory requirements which aim

at protecting sensitive or personal data as well as general public security requirements impel

them to devote the utmost attention and priority to information security risks. If a service is not

tested then there will be no information about its security or insecurity. A security test is unlikely

to provide information about new vulnerabilities, especially those discovered after the test is

carried out. Vulnerability assessments that include careful diagnostic reviews of all servers and

network devices will definitely identify more issues faster than a "black box" test.

The chief objective of work is to implement the appropriate measurements in order to eliminate

or minimize the impact that various security related threats and vulnerabilities (i.e. security

auditing) might have on an organization.

17

6. References

[1] Ramy K. Khalil, Fayez W. Zaki , Mohamed M. Ashour, and Mohamed A. Mohamed, “A study of

network security systems,” IJCSNS International Journal of Computer Science and Network Security, VOL.10 No.6, June 2010, pp. 204–212.

[2] Mr. V. K. Pachghare and Dr. Parag kulkarni, “Network security based on pattern matching: an overview,” IJCSNS International Journal of Computer Science and Network Security, VOL.8 No.10, October 2008, pp. 314–318.

[3] Kapil Kumar Gupta, Baikunth Nath (Sr. Member IEEE) and Kotagiri Ramamohanarao, “Network

security framework,” IJCSNS International Journal of Computer Science and Network Security, VOL. 6 No.7B, July 2006, pp. 151–157.

[4] Kulvinder Singh, Rakesh Kumar and Iqbal Kaur, “Testing web based applications using finite state

machines employing genetic algorithm,” Faculty of Computer Science & Engineering, published in International Journal of Engineering Science and Technology, Vol. 2(12), 6931-6941, 2010.

[5] Hazlifah Mohd Rusli, Suhaimi Ibrahim and Mazidah Puteh, “Testing web services composition: a

mapping study,” IBIMA Publishing, Communications of the IBIMA, Article ID 598357, Vol. 2, 12 pages, 2011.

[6] Mohd. Ehmer Khan, “Different forms of software testing techniques for finding errors,” Department

of Information Technology, Al Musanna College of Technology, Sultanate of Oman, IJCSI International Journal of Computer Science Issues, Vol. 7, Issue 3, No 1, May 2010.

[7] Los Alamitos, CA: IEEE Computer Society Press, “IEEE Standard Glossary of Software Engineering Terminology,” IEEE Std 610.12- 28 sep 1990.

[8] Young Gun Jang, Hoon Il Choi and Chan Kon Park, “Implementation of home network security

system based on remote management server,” IJCSNS International Journal of Computer Science and Network Security, VOL.7 No.2, February 2007, pp. 267–274.

[9] W Makasiranondh, S P Maj and D Veal, “An integrated multimedia based platform for teaching network security,” IJCSNS International Journal of Computer Science and Network Security, VOL.10 No.12, December 2010

[10] Igor Kotenko and Mikhail Stepashkin, “Analyzing network security using malefactor action graphs,”

IJCSNS International Journal of Computer Science and Network Security, VOL.6 No.6, June 2006, pp. 226–235.