1

Test Automation

Using

SilkTest

2

Test Automation Rules

• Product feature understanding, test planning, test case documentation, test bed setup, defect tracking, progression tests are all manual

• Test automation success depends on test cases

• Every manual step has commands in test script

• If it is not worth, do NOT automate

3

Tools - Comparison• WinRunner, QTP are from Mercury Interactive – USD 1bn company and market

leader; Robot from Rational IBM - USD 100bn company’ Silk from Segue – USD 50mn company

• Script here is object oriented and close to QTP script. Winrunner TSL is like C and Robot SQA Basic is like Visual Basic

• GUI to create verification points are better in WinRunner, QTP and Robot• Silk is more towards moving scripts on MS Windows and Unix; whereas Robot or

WinRunner scripts cannot be moved to other OS as those tools do not support• Rational does not provide GUI specification in a separate file; WinRunner, QTP and

Silk provide that• You need to buy Java Plugin for WinRunner; Silk and Robot come with Java plugin• Record, replay, verification method, data driven tests, database connectivity, bitmap

verification are all same at concept level and differ only at script syntax level• Wizards are less in SilkTest, compared to other tools• Cost-wise, QTP and WinRunner are most expensive; Silk stands in between

WinRunner and Rational Robot

4

5

Architecture

• Project .vtp (for maintenance)• Testplan .pln (organizing the tests)• Test Frame* .inc (AUT Window details)• Scripts .t (test scripts)• Data driven tests .g.t (scripts with many data)• Include file .inc• Suite .s (for batch runs)• Result .res

6

Test Automation Principle

• Make the tool learn about AUT GUI• Record steps based on mouse and key events• Organize steps into test case• Organize test cases into test plans• Run test cases• Batch different sets and run• Analyze results• Do NOT try to code application logic into testing code• Whenever possible, generate code thru the tool and not

hand-code• Read STUSER Help file (user guide for SilkTest)

7

Test Process

• Manage requirements using SilkCentral Test Manager

• Manage test plans and cases using Silk test • Associate test plans/cases to requirements

using test manager• Declare GUI declarations using SilkTest• Develop scripts for atomic activities first so that

they can be reused later• Develop scripts using SilkTest and run them• Track defects using Defect Tracker

8

Test Frames• Before test case recording, do this step• This creates an .inc file that has GUI declarations• Identifies GUI controls as classes• Label or caption is the best way to identify objects; if not available,

object index is used (available tags are caption, prior text, index, window id, coordinates)

• Class and tag are the key elements in declarations• Record window declarations is the way to store test frames• When objects are not identified properly, use different tags to

uniquely identify the objects. This will vary from application to application

• If object X,Y coordinates are to be used, declare constants and then use those constants for further declarations

• To declare new objects within a window, it is better to use AnyWin or CustomWin classes. Use ~ActiveApp to specify the coordinates

9

10

Test Frames• Along with window declarations, we can declare methods that are

specific to that window and objects• The method declarations within a window object is similar to object

oriented programming• Uses can use these method declarations as wrapper classes for

executing a series of steps that are recorded• To the extent possible, do not have any hard coded values in the

GUI Window methods• While recording new test cases, ensure that you update window

declarations as well; but check the frame.inc file, not to have duplicate declarations for same window

• Default is frame.inc; but we can have any number of include files• Unless the window declarations are in scope, the test script will not

recognize the controls

11

Recording• Recording will automatically generate statements on every user

action• To capture the mouse click coordinates and keyboard events, we

can use low level enabler for every control in recording options• We can record the base state as well as test steps• Soon after recording, ensure the test frame declarations, for any

changes or duplicates• Recording can be done for window declarations or test case or

actions • Every recorded script must start from a known base state and must

return back to the same known base state• BaseState and DefaultBaseState can also be specified using

recording (appState is a function that ensures the pre-condition state for a test case)

12

13

14

Test Cases• Each test case must be independent• Initialize (pre-requisite setup), run and verify, cleanup• One test case must have One purpose• Establish application state for every test case. Record the base

state• Test Case contains commands for every test step. Record the

steps and edit if needed• Establish verification points for every test case• Record the clean up steps as well• Sequencing of test cases is very important when it comes to

automation• These are also functions, with the type as “testcase”• We can use parameters for testcase as well• A testcase is also a function by itself, but it can have base states

15

4Test Elements• Variables – INTEGER, STRING, BOOLEAN, ANYTYPE, REAL, LIST• Variables do have scope – public or private• Const is for constants• Regular arithmetic and string operations hold good• 2 types of for statements• Foreach and While loops are also available• IF, select-case and SWITCH-case are available for branching• Do-except is for exception handling• Exception functions will be useful for tracing exception number and details• Test steps will have window.object.method or window.object.property style

of commands• User defined functions with in, out, inout parameters• Functions can return values as well• Use built-in functions for list and array manipulations• Print and Printf commands are used for sending output to results file

16

Built-in Functions, Classes• For debugging purpose, we can use print function. It helps in

documentation as well as to track the test run status, at any given point of time

• Refer to General Core Classes section in SilkTest Help (PARTNER file) – AnyWin, MainWin, CheckBox, ComboBOx, DialogBox, ListBox, Menu, PushButton, RadioButton, StaticText, TextField, StatusBar are the most used classes

• Refer to Functions by Programming Task section in SilkTest Help (PARTNER file) – Datatype commands, Database commands, List and string Manipulations, Window Information commands are the most used ones

• System related functions can be used to carry out any DOS commands to set the test environment (Sys_Execute, Sys_Kill)

• String functions such as ltrim, rtrim, etc are used extensively to manipulate test data; arithmetic functions are not used often

• User defined functions must be called from main function

17

18

19

AnyWin Properties & Methods• CaptureBitmap• ClearTrap• Click• DoubleClick• Exists• GenerateDecl• GetAppId• GetArrayProperty• GetBitmapCRC• GetCaption• GetChildren• GetClass• GetContents• GetEverything• GetHandle• GetHelpText• GetIDGetIndex• GetInputLanguage

• GetManyProperties • GetName• GetNativeClass• GetParent• GetProperty• GetPropertyList• GetRect• GetTag • InvokeMethods • IsActive• IsArrayProperty• IsDefined• IsEnabled• IsOfClass• IsVisible • InvokeJava • MenuSelect • MoveMouse • MultiClic

20

AnyWin Properties & Methods• MultiClick• PopupSelect• PressKeys• PressMouse• ReleaseKeys• ReleaseMouse• ScrollIntoView• SetArrayProperty• SetInputLanguage• SetProperty• SetTrap• TypeKeys• VerifyActive• VerifyBitmap• VerifyEnabled• VerifyEverything• VerifyText • VerifyProperties• WaitBitmap

• bActive• AppId• sCaption• lwChildren• Class bEnabled• bExists• sID• iIndex• sName• wParent• Rect• hWnd• WndTag

21

Method examples• Find.OK.Click• Find.MatchCase.Check• Find.FindText.SetText(“abc”)• Notepad.CloseWindows• Find.Up.VerifyValue(TRUE)• Fonts.FontName.verifyContents(…)• Notepad.Editor.SelectAll() // this is a custom method to select all

contents. This needs to be declared in the windows declarations• User defined functions can have in, out, inout and optional

parameters• ClipBoard functions are used to manipulate clipboard text• Cursor class methods can be used to manipulate coordinates• Property prefixes – b for boolean, I for int, ls for list, s for string

22

Get, Set, Verify for Controls• For every object, Get methods are used to obtain attribute values into a

variable. After getting to a variable, we need to manipulate• For every object, Set methods are used to set attribute values on the

screen. This will work only when the object cooperates with SilkTest operations

• For every object, Verify methods are used to check the attribute values – implicit Get and an if-then-else condition. It must have an expected value as argument

• Whatever properties that can be used for get statement, can be used for set and verify statements

• Verify methods evaluate to pass or fail status• Any other custom methods can be used, if they are already declared in

window declarations• Most of the properties and methods are inherited from AnyWin to all other

GUI classes• Dlls can be loaded using dll declaration and the functions can be invoked

from 4Test• Capture and Compare bitmap commands are used for image checks

23

Utility Elements• Array functions are used to manipulate arrays (ArraySize, ArrayFind)• List functions are used to manipulate lists (ListSort, ListAppend)• DateStr, TimeStr, Sleep, StartTimer, StopTimer are used for timer functions• FileOpen, FileClose, FileWriteLine, FileReadLine are used for file operations• Wild cards - ? And *; MatchStr command is used for expressions• Multitags enable an object to be identified with many labels• Registry Operations are available – not used widely (Reg_CreateKey)• LogError, LogWarning, Print to write to results file• Timer commands to create, start, stop, destroy and get the interval• Date format and functions to manipulate date fields• DLLs can be loaded and then functions can be executed; but this needs to

be kept at minimum• To get script, environment and browser related information, we can use the

respective built-in functions (GetArgs, GetBrowserType)• Date related functions are available; but it is advised not to use inside script

to manipulate dates

24

Database Test

• Database tests follow the verification of GUI verifications

• Database commands start with DB_• Connect to DB (DSN needed)• Execute Query (give SELECT)• Fetch the records and compare with baseline• Finish DB operation (release resources)• Disconnect after every database operation• Do not keep the database connections for a long

time in 4Test scripts

25

Verification

• Object properties are the key elements• Control state, value, label, text, count, size,

location are the most common properties• Feed the expected result to the verify command• Bitmaps can be verified using CaptureBitmap

and VerifyBitmap• Use Bitmap tool to capture baseline image• Is* methods evaluate to a boolean value

(IsActive, IsEnabled)• LinkTester checks the links in web applications

26

Data Driven Tests

• Same script, variety of data, less coding• Use Data Driven work flow for a standard

recorded test case• Create a test database and configure DSN• Replace hard coded recorded values to data

driven variables (using find/replace option)• When you run the test, it runs for all rows of data

in the data set• A new test case with DD_ prefix will be created

27

28

Run Tests

• One can run a test case or test plan or test suite• Suite contains many test cases in it• Using distributed run mode, one can run scripts in

another machine that has Silk Agent• We can run in run mode or debug mode• Debug mode enables us to step, step into, step out,

declare breakpoints etc• All other hosts run the Agent• Spawn, Parallel, Rendezvous are used in distributed

testing – but unless required, distributed testing is not widely used

• One must connect and disconnect to target machines

29

Traceability

• Requirements to test traceability is thru Test Manager• Test case to tester traceability is again part of test

manager • We need to define naming conventions by ourselves for

test plans, test cases, requirements• Suggested guideline is to name the test case with the

test case ID itself and have the test case description as comments

• Use base states for capturing test pre-requisites• Version control is integrated with PVCS for check-in,

check-out and labeling features

30

Recovery Scenarios

• Recovery system enables the test to come to the Base State

• Record application state is used to specify the recovery scenarios

• Exception handler itself is script recovery feature

• StopRunning() method bypasses all recovery settings

31

Test Extensions

• These are used to help SilkTest to identify new objects and their properties

• Silktest can automatically enable extensions for open applications

• Java, .Net, AOL Browser, Netscape Navigator, Internet Explorer require extensions

• For browsers, enabling DOM or Virtual Object mode would be configured to identify objects

32