test automation with twist and sahi
DESCRIPTION
A brief presentation on Twist, Sahi, and design patterns that assist with GUI Functional Test Automation.TRANSCRIPT
Test Automation with Twist and Sahi
May 2014
Mattersight Corporation
1
Eric Blackburn, Chris Stanush, Peter Zejda
Agenda
2
• Why• Sahi• Sahi vs Selenium• Twist• Design Suggestions• Demo Video• Questions
Save Questions Till The End
3
• We will stick around
Agenda
4
• Why• Sahi• Sahi vs Selenium• Twist• Design Suggestions• Demo Video• Questions
Why
5
• Functional Test Automation – hard
• GUI Functional Test Automation - especially hard− DOM changes occur often− Complex HTML controls − Support multiple browsers− Record and playback is fragile – doesn’t support change− Need for separating the “What” from the “How”
• Intent (the “What”) doesn’t change as often as the testing method (the “How”)
• Business side needs to write/maintain scenarios, not developers
• Automated testing needs to be faster than manual
• Testing tools can be expensive
• Need ability to “check in” / version Test Cases with code
• Need a full language and enterprise software architecture / design to create an effective test framework (and make part of ConDel)
Agenda
6
• Why• Sahi• Sahi vs Selenium• Twist• Design Suggestions• Demo Video• Questions
7
What is Sahi
• Sahi is the proxy between Java and the Browser− Java -> Sahi -> Javascript -> Browser− Java <- Sahi <- Javascript <- Browser
• Learn more about Sahi: http://sahi.co.in/
Agenda
8
• Why• Sahi• Sahi vs Selenium• Twist• Design Suggestions• Demo Video• Questions
9
Sahi vs Selenium
• Selenium− Positives
• Run JavaScript directly against the browser• Widely used• Different flavors
− Selenium 2.0− Geb− Helium
− Negatives• Lack of auto-waits for Ajax calls
− http://www.thoughtworks.com/products/docs/twist/current/help/how_do_i_handle_ajax_in_selenium2.html
• Wait for driver updates• For built in functions, different experiences with different drivers.
• Sahi− Positives
• Run JavaScript directly against the browser• Automatically waits for page to load and for any Ajax calls to finish• For built in functions, same experience for all browsers.• Supports concurrent testing (batch runs)
− Negatives• Sometimes proxy doesn’t initiate properly.
Agenda
10
• Why• Sahi• Sahi vs Selenium• Twist• Design Suggestions• Demo Video• Questions
11
Twist http://www.thoughtworks.com/products/twist-agile-testing
• BDD interface built into Eclipse
• Cost − $100 a year per developer
• Free updates
− Able to export libraries and run on test harness systems/servers for free, as many test harness servers as desired.
12
Twist - Behavior-driven Development BDD [1]
• Domain Specific Language (DSL)• Business can write tests
− Non-formal English− Specify intent− Abstracted step from how it is implemented
• Code changes, but the sentence (intent) remains the same.
− Prior to code being implemented• BDT
[1] http://en.wikipedia.org/wiki/Behavior-driven_development
13
Twist
• GUI and Non-GUI functional testing• Contexts
− Setup and Teardown steps always run, when added to a scenario
• Tags− Used to organize scenarios− “Not IE7”, “Disabled”, “Active”
• Steps from multiple Classes• Comments• Manual steps• Table driven steps• No browser focus needed• Auto snapshots for failures• Auto generated HTML result reports• Jenkins slave runs tests via ANT or Maven
− Windows needs VNC service running (browser needs GUI rendering). Real VNC free edition works fine.
− Reportedly Linux can run headless
14
Twist
• Table driven steps−Can fail sections of the table using Asserts
• Run results as HTML Report• More info about how to use various features is available at
− http://www.thoughtworks.com/products/docs/twist/current/help/whats_new.html
Agenda
15
• Why• Sahi• Sahi vs Selenium• Twist• Design Suggestions• Demo Video• Questions
16
Design Suggestions – Browser Adapter
• Problem− Dependency on the web interaction
tool (Sahi)
• Solution− Abstract browser interaction
• Portability• Identify elements via JavaScript
chaining (pass around strings)
− Browser Adapter class • All of code inherits added or changed
classes • Doesn’t help when a class is removed
− Create own API• Not a big deal
− 15 methods for Action− 7 methods for State
• Better API than original
17
Design Suggestions – Control Object Pattern
• Problem− Many web controls
• Each with specialized find, set and verify functionality
− Control interfaces can be very complicated
− Control interfaces do change
• Solution− Follow the “Information Expert”
design pattern[1] • Encapsulate logic for
− Find and Set methods − Verify state of control
[1] http://en.wikipedia.org/wiki/GRASP_(object-oriented_design)#Information_Expert
18
Design Suggestions – Pre-build JavaScript/jQuery
• Problem− Calls to the browser are
slow
• Solution− Move logic into JavaScript
• Browser is able to process complex logic fast
• Sometimes the algorithm needs to be next to the data
19
Design Suggestions – Jsoup http://jsoup.org/
• Problem− Need for complex inspection of DOM and calls to the browser are
slow.
• Solution− Use Jsoup to cache the DOM
• Return DOM content as a String and parse directly in code• Complex processing of the DOM is fast • Syntax is easy to use
20
Design Suggestions – Find Elements
• Problem− Identify web content based on what the scenario writer can see
• Solution− public String findNthVisibleElement(FieldTypeEnum fieldType, String
parentSelectorValue, String elementIdentifier, int index)
• A central function that is able to find elements. Elements can be identified by their text content, id, name, aria-label, etc.
− Identify an element by passing around a String that can be used to scope up and down• Use JavaScript or jQuery syntax
21
Design Suggestions – Find Elements – Helper Methods
• List out what controls can exist as
• List out what identifies are used for each HTML element type
22
Design Suggestions – Find Elements – Helper Methods
• Build comma delimited selector using the identifier and the list of element types− e.g. “#userinput, .userinput, [aria-label=‘userinput’]”
23
Design Suggestions – Find Elements – Helper Methods
• ContainsExact(Case) – Find element that only contains• Problem - :Contains(‘some of text’) doesn’t work
• Matches both− <span>some of text</span>− <span>text and some of text and more text</span>
• Solution - :ContainsExact(Case)(‘some of text’) works• Matches only
− <span>some of text</span>
Agenda
24
• Why• Sahi• Sahi vs Selenium• Twist• Design Suggestions• Demo Video• Questions
• Concerns, Ideas, Requests, Champagne?
Questions?