AUTOMATED USER ACCEPTANCE TESTING OF WEB APPLICATIONS WITH SELENIUM
Presented by Onur Yaman 2010719258 Bogazici University – Software Engineering
Overview • General Definitions • Problem Statement • Solutions Proposal • Why Selenium? • What is Selenium? • Demo • What’s next? • References
2
General Definitions
• User Acceptance Testing
• Final testing based on
• Specifications of the end-users or customer
• Use by end-users or customer over some limited period of time
• Completed system vs. requirements
• Verification testing
• Validation testing
• Audit testing
• Performed by users of the system
1/3
3
General Definitions
• Test Automation
• Use of software to control the execution of tests
• Comparison of actual outcomes to predicted outcomes
• Setting up
• Test preconditions
• Test control and reporting functions
• Automating a manual process already in place that uses a
formalized testing process
2/3
4
General Definitions
• Web Application
• Application that is accessed over a network
• Client-server architecture
• Ability to update and maintain without distributing and installing on
potentially thousands of clients
• Cross-platform compatibility
3/3
5
Problem Statement
• User acceptance testing of web applications is
• Costly
• Browser (client) compatibility
• Web applications tend to get updated frequently
• Many possible inputs
• Slow
• Tied to browser performance
• Clients and servers communicate over network
6
Solution • Automating user acceptance tests
• Usage of proprietary software
• Many tools exist
• FitNesse
• HtmlUnit
• HttpUnit
• Watir, Vapir, Watij, WatiN, Win32-Watir, Watir for Rails
• Celerity
• iMacros
• Canoo WebTest
• Selenium
7
Why Selenium? • Handles JavaScript well
• Other tools are not reliable in handling in-page JavaScript
• Provides maintainable tests • Other tools tend to break after small page changes
• Works well with TDD
• Easy-to-write scripts
• Open source • Possibility of extending
• Supports many browsers
• Supports many programming languages
8
What is Selenium? • Automated acceptance testing tool • Developed by ThoughtWorks, now open source • Portable
• Deploys on Windows, Linux and MacOSX • Supports various programming languages including C#, Java, Python,
PHP, Perl, Ruby, Groovy • Supports most modern web browsers including Firefox, Safari, Google
Chrome, Opera, Internet Explorer • Supports popular smart phones like Android and iPhone
• Tool suite • Selenium 1 (a.k.a. Selenium RC or Remote Control) • Selenium 2 (a.k.a. Selenium WebDriver) • Selenium IDE • Selenium Grid
1/10
9
What is Selenium? • Selenium 1 (a.k.a. Selenium RC or Remote Control)
• Was the main Selenium project
• Still actively supported
• Supports several languages • Java, JavaScript, Ruby, PHP, Python, Perl, C#
• Supports almost every browser out there
• Composed of components • Selenium Server
• Manages browsers, runs Selenese commands, acts as HTTP proxy between the browser and the AUT
• Client libraries • Provide interface between any programming language and the Selenium Server
2/10
10
What is Selenium?
• Selenium 1 (a.k.a. Selenium RC or Remote Control)
• Architecture
3/10
11
What is Selenium? • Example for Selenium 1 (a.k.a. Selenium RC or Remote Control)
4/10
1- Run the Selenium Server
2- Run the language specific tests using the corresponding client API
12
What is Selenium?
• Selenium 2 (a.k.a. Selenium WebDriver)
• Provides a simple, concise programming interface
• Better supports dynamic web pages (AJAX)
• Supplies a well-designed object-oriented API
• Makes direct calls to the browser using browser’s native support for
automation (browser dependent)
• Possibility of not using Selenium-Server
5/10
13
What is Selenium? • Example for Selenium 2 (a.k.a. Selenium WebDriver)
6/10
14
What is Selenium? • Selenium IDE
• Tool to develop Selenium test cases
• Uses Selenese (Selenium script language)
• Firefox plug-in
• Allows saving macros (like Microsoft Office macros)
• Allows exporting saved test cases to any desired language that uses
the Client API
• Allows grouping test cases as test suites
• Allows running previously saved test cases/suites
• Provides results of tests
7/10
15
What is Selenium?
• Example for Selenium IDE
8/10
16
What is Selenium? • Selenium Grid
• Builds on the traditional Selenium setup • More than one machine can be used
• Selenium RC and the browser are not tied to a specific application (they can be shared by multiple applications and projects)
• Possibility of a distributed grid system of Selenium RCs • Shared across builds, applications, projects, organizations
• Scalable • Selenium Hub
• Allocates a Selenium RC to a specific test (transparently)
• Limits the number of concurrent test runs on each RC
• Shields the tests from the actual grid infrastructure
9/10
17
What is Selenium?
• Selenium Grid Setup
10/10
18
Demo
• Live demo
• Bingo! – Java Web Application
• Very simple test cases
• Selenium WebDriver is used
• Implementation is made in Python 2.7
19
What’s next? • Writing good tests
• Keep tests self-contained
• Write own extensions if needed
• Write tests before development (TDD)
• Refactor tests (if possible)
• Selenium and FitNesse Integration • Everyone (including the customers) see, track and run tests
• Continuous Integration • Running selenium tests is possible (in-memory X window system)
• Test coverage analysis
20
References • Selenium Documentation (Release 1.0) / May 10, 2012
<URL: http://seleniumhq.org/docs/book/Selenium_Documentation.pdf>
• Selenium Grid: How It Works / 2008 <URL: http://selenium-grid.seleniumhq.org/how_it_works.html>
• Holmes, A.; Kellogg: Automating functional tests using Selenium. M.Agile Conference, 2006.
• Bruns, A.; Kornstadt, A.; Wichmann, D.: Web Application Tests with Selenium. Software, IEEEVolume: 26 , Issue: 5.
• Razak, R.A.; Fahrurazi, F.R.: Agile testing with Selenium. Software Engineering (MySEC), 2011 5th Malaysian Conference in Digital
Object Identifier: 10.1109/MySEC.2011.6140672.
21