implementing test automation in agile projects
DESCRIPTION
TRANSCRIPT
Implementing Test Automation in Agile Projects
Michael Palotas & Dominik DaryQuality Engineering Europe
eBay International
April 2012
Founded in 1995
Based in San Jose, California
27770 Employees worldwide
eBay Inc. Facts
104 million active buyers and sellers worldwide
68 Billion USD transaction volume 2011
2000 USDtransaction volume every second
Platform Facts
9 petabytes of dataIn our Hadoop and Teradata clusters
2 billion page viewseach day
75 billion database callseach day
Data Facts
70,000,000downloads of the eBay Inc. mobile apps
5 billion USDtransacted 2011 on mobile devices
1 million itemsare listed with using mobile apps every week
Mobile Facts
Who we are
Michael PalotasHead of Quality Engineering Europe
E-mail: [email protected]
Dominik DaryStaff Software Engineer in Test
E-mail: [email protected]
To get early and instant Feedback
We automate the regression tests
It is our safety net to save time & this is reinvested into manual testing
Test Automation?
Inspiration: http://patrickwilsonwelsh.com/?p=32
Flip the testing triangle
Unit Tests
E2E AutomationTests
Manual Tests
Unit Tests
E2E
Tests
Manual Tests
Integration Tests
Our guide through this Presentation
2. Modeling of the Biz Domain Layer
3. Test Implementation
1. Design the Automated Test with Test Aspects
4. Lean Test Automation
**Swiss Testing Day Presentation 2010: “Mit Testaspekten früh, intelligent und redundanzfrei testen” Melanie Späth
Image Source: http://www.masternewmedia.org/images/social_software_impact_individual_organizations.jpg
1. Design the Automated Tests with Test Aspects
Use Test Aspects**
• Test aspects briefly describe “why” and “what” has to be tested, without describing “how”
• Enables for early testing because tests can be written at the same time the user story is created
• Test aspects are structurally derived
Advantages
• Clear overview for all team members about test coverage
• Simplicity helps you to do quick reviews and walkthroughs with your product owner
• Enables you to identify manual and automated test duplicates
• Traceability between test cases and specification
Test Aspect Example
2. Modeling of the Biz Domain Layer
Biz Domain Layer
Browser
Flow Objects
Tests
WebDriver
Page Objects
BizDomainLayer
• Modularization and reuse of components: maintainable & extendable e2e tests
• Crucial to have in depth knowledge of the domain
• Enable data driven testing by generic test cases
e-Commerce shopping site like eBay
Checkout Page Success PageShopping Cart
PageView Item
Page
Pages and Flows
ViewOrderlist
ViewCategory
Update
Delete ViewProduct
SearchItem
ViewCategory
ReviewSelect
Payment Provider
OrderSummary
3. Test Implementation using SeleniumTe
sts
Flo
w O
bjec
tsP
age
Obj
ects
Technical High Level Overview
Instant Feedback - Selenium Grid
Sequential Execution
Execution time:7000 minutes (~116 hours = ~5 days)
Test 1
Test 2
Test …
Test4500
Execution Time
Test 3
Footnote about the size of the Tests @eBay Europe: - 4500 end-to-end regression test cases- 2-3 minutes execution time for each
Parallel Execution
Execution time: 60 minutes
by using 150 Virtual Machines / Browsers using the WebDriver GRID
Test Test Test
Execution Time
Test
Test Test TestTest
Test Test TestTest
Pa
ralle
l Exe
cutio
n
Pa
ralle
l Exe
cutio
n
Definition of Done
Reviews
Document the essentials
Don’t build a framework
Long Leash
Lean Test Automation• Eliminate Waste
• Build Quality In
• Create Knowledge
• Defer Commitment
• Deliver Fast
• Respect People
• Optimize the Whole
4. Lean Test Automation
Technical Debt• Activities that result in
technical debt include• Postponing
documentation and writing tests
• attending to TODO comments and
• tackling compiler and static code analysis warnings
• Knowledge that isn't shared around the organization
• Code that is too confusing to be modified easily
Be clear why you want to automate
Conclusion
Test automation is much more than
just your automation toolset!
Open source is your best friend
Flip the testing triangle
Treat automation as a software project
Work with the right people
Stay lean
Thank You! Questions or Comments?