© Copyright 2010, Accellence (Thailand) Ltd.

Automated Testing for Web application

June 21, 2012

Kiattikhun Prathumma

© 2010 Accellence (Thailand) Limited.

Agenda

2

• What is Automated Testing

• Automated Testing vs Manual testing

• Automated Testing Framework

• Demo

• Q&A

© 2010 Accellence (Thailand) Limited.

What is Automated Testing

3

“…Test automation is the process of writing a

computer program to do testing that would

otherwise need to be done manually…”

Source: http://en.wikipedia.org/wiki/Test_automation

© 2010 Accellence (Thailand) Limited.

Advantages

4

Automated Testing Manual Testing

If you have to run a set of tests repeatedly automation is a huge gain

If Test Cases have to be run a small number of times it's more likely to perform manual testing

Helps performing "compatibility testing" - testing the software on different configurations

It allows the tester to perform more ad-hoc (random testing)

It gives you the ability to run automation scenarios to perform regressions in a shorter time

Short term costs are reduced

It gives you the ability to run regressions on a code that is continuously changing

The more time tester spends testing a module the grater the odds to find real user bugs

Can be run simultaneously on different machines thus decreasing testing time

Long term costs are reduced

© 2010 Accellence (Thailand) Limited.

Disadvantages

5

Automated Testing Manual Testing

It's more expensive to automate. Initial investments are bigger than manual testing

Manual tests can be very time consuming

You cannot automate everything, some tests still have to be done manually

For every release you must rerun the same set of tests which can be tiresome

© 2010 Accellence (Thailand) Limited.

Use Automated Testing?

6

“…It only makes sense to use automated

testing tools when the costs of acquiring the

tool and building and maintaining the tests is

less than the efficiency gained from the effort"

John Overbaugh, Senior Lead Software Development Engineer In Test (SDET) at Microsoft

© 2010 Accellence (Thailand) Limited.

Use Manual Testing?

7

“…Once a team is good at managing testing risk

and test coverage and applying the right testing

techniques, then talking about automation makes

sense."

Mike Kelly, Software development manager for a Fortune 100 company

© 2010 Accellence (Thailand) Limited.

Other factors

8

• The performance of your test tools

• The knowledge level of your testing team

• The continuous growth of software to be tested

• Number of necessary regressions

© 2010 Accellence (Thailand) Limited.

General approaches to test automation

9

1. Code-driven testing- Testing frameworks such as the xUnit

frameworks

2. Graphical user interface testing- A testing framework generates user interface

events such as keystrokes and mouse clicks, and observes the changes that result in the user interface

© 2010 Accellence (Thailand) Limited.

Automated Testing Framework

10

Tool name Produced by Latest version

HP QuickTest Professional HP Software Division 11.0

HTTP Test Tool Open source 2.0.8

IBM Rational Functional Tester

IBM Rational 8.2.1

LabVIEW National Instruments 2011

Maveryx Maveryx 1.2.0

Oracle Application Testing Suite

Oracle Corporation 12.1

QF-Test Quality First Software GmbH 3.4.7

Ranorex Ranorex GmbH 3.2.3

Rational robot IBM Rational 2003

Selenium Open source 2.20

SilkTest Micro Focus 2011

© 2010 Accellence (Thailand) Limited.

Automated Testing Framework (Cont.)

11

Tool name Produced by Latest version

eggPlant TestPlant 2012

EiffelStudio AutoTest Eiffel Software 7.0, Nov 2011

SOAtest Parasoft 9.2

TestComplete SmartBear Software 8.7

Testing Anywhere Automation Anywhere 7.5

TestPartner Micro Focus 6.3

TPT PikeTec GmbH 4.0

TOSCA Testsuite TRICENTIS Technology & Consulting 7.3.1

Visual Studio Test Professional

Microsoft 2010

WATIR Open source 3.0

Telerik Test Studio Telerik, Inc. 2012.1

© 2010 Accellence (Thailand) Limited.

Watir

12

• It’s a Ruby library

• Multi browser (& OS) support

• Has a rich API

• Has a ‘Simple’ class (for non-tech users)

• Watij & Watin (Java & .NET)

© 2010 Accellence (Thailand) Limited.

Watir Users

13

Source: http://wiki.openqa.org/display/WTR/Watir+Users

© 2010 Accellence (Thailand) Limited.

What kind of automated testing does Facebook do?

14

© 2010 Accellence (Thailand) Limited.

Watir at Facebook

15

* Steven Grimm, Facebook’s test engineering tech lead

© 2010 Accellence (Thailand) Limited.

Watir Demo

16

© 2010 Accellence (Thailand) Limited.

Selenium

17

• Multi browser, OS & language support

• Install server-side or as FF add-on

• Has its own IDE

• Record and playback tests

© 2010 Accellence (Thailand) Limited.

Selenium Demo

18

© 2010 Accellence (Thailand) Limited.

Q&A

19

Problem?

© 2010 Accellence (Thailand) Limited.

Reference

20

• http://www.slideshare.net/vivek_prahlad/web-test-automation-with-selenium

• http://www.slideshare.net/sebastian_bergmann/testing-phpweb-applications-with-phpunit-and-selenium

• http://davidweiss.blogspot.com/2006/08/when-to-automate-testing.html

• http://en.wikipedia.org/wiki/Test_automation

• http://en.wikipedia.org/wiki/List_of_GUI_testing_tools

• http://www.fitnesse.org/FrontPage

• http://seleniumhq.org/

• http://www.chapterpiece.com/software-development-process/2010/02/06/choosing-automated-testing-or-manual-testing/

• http://www.welovebug.com/software-testing/manual-vs-automated-testing/

• http://www.chapterpiece.com/software-development-process/2010/01/21/tell-us-why-automated-testing-failed/

• http://www.chapterpiece.com/software-development-process/2010/01/30/why-automated-testing-fails/

• http://www.chapterpiece.com/software-development-process/2010/01/03/testing-in-agile/

• http://blogs.mulesoft.org/web-testing-selenium/

• http://watir.com/

• http://www.outsourcebazaar.com/index_Article_AutomatedVsManualTesing.html

• http://www.slideshare.net/directi/automated-testing-vs-manual-testing-391121

© 2010 Accellence (Thailand) Limited.

Reference (Cont.)

21

• http://allfacebook.com/how-facebook-automatically-tests-code-phpunit-watir_b15369

• http://www.quora.com/Facebook-Engineering/What-kind-of-automated-testing-does-Facebook-do

• http://www.slideshare.net/sumanth/watir-presentation-sumanth-krishna-a

• http://java-thai-talk.blogspot.com/2011/01/automated-testing-vs-manual-testing.html

• http://googletesting.blogspot.com/2007/10/automating-tests-vs-test-automation.html

• http://www.hanselman.com/blog/NewReleaseOfWatirMakerNowWatirRecorder.aspx

• http://wiki.openqa.org/display/WTR/Example+Test+Case

• http://blog.sahi.co.in/2010/04/sahi-vs-selenium.html