strategies for selecting the right open source framework ... · headless browsers role motivation...
Post on 06-Jul-2020
2 Views
Preview:
TRANSCRIPT
BW6 Test Automation Wednesday, June 6th, 2018, 1:30 PM
Strategies for Selecting the Right Open Source Framework for Cross-Browser
Testing
Presented by:
Eran Kinsbruner Perfecto
Brought to you by:
350 Corporate Way, Suite 400, Orange Park, FL 32073 888-- -268- - -8770 ·· 904- --278-- -0524 - info@techwell.com - https://www.techwell.com/
Eran Kinsbruner Perfecto Eran Kinsbruner is the mobile evangelist at Perfecto and the author of the digital quality handbook, as well as a monthly columnist at InfoWorld.com. He is a software engineering professional with nearly twenty years of experience at companies such as Matrix, Qulicke & Soffa, Sun Microsystems, General Electric, Texas Instruments, and NeuStar. He holds various industry certifications, including from ISTQB and CMMI. Eran is a recognized mobile testing influencer and thought leader, as well as an experienced speaker at the major software engineering conferences. He is also a patent-holding inventor (test exclusion automated mechanism for mobile J2ME testing), researcher, and blogger. Eran can be found all over social media, including on Facebook, Twitter (@ek121268), LinkedIn, and his professional mobile testing blog.
5/26/18
1
Strategy For Selec3ng the Right Cross—Browser Tes3ng Tools
About Me • Lead Technical Evangelist at Perfecto • Blogger and Speaker
• http://continuoustesting.blog • https://www.infoworld.com/author/Eran-
Kinsbruner/ • 18+ Years in Development & Testing • Author of The Digital Quality Handbook
Weekly Podcast -‐ Tes3umpod
Twi1er: @ek121268 Email: Erank@perfectomobile.com
5/26/18
2
Agenda
Ø A word about the digital transformation Ø RWD & PWA Testing As The De-Facto Web App Types Ø Cross-Browser Testing Landscape and Criteria Ø Q&A
5/26/18 3 © 2015, Perfecto Mobile Ltd. All Rights Reserved.
Lack of Mobile Capabilities in Web Browser
Camera Mic
Device auth
BT/BLE
No3fica3on Network
5/26/18
3
Cross-Browser Testing != Desktop Web Testing
There is no Web Testing vs. Mobile Testing
• 4 out of 10 transac3ons today take place on mul3ple devices
• 48% of users today complain that the websites they use are
not op3mized for their smartphones and tablets
DevOps == Continuous Everything
5/26/18 6 © 2015, Perfecto Mobile Ltd. All Rights Reserved.
Credit: Tes3ngWhiz
5/26/18
4
Fundamentals of RWD Testing
© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Drivers & Benefits To Moving to RWD
5/26/18 8 © 2016, Perfecto Mobile Ltd. All Rights Reserved.
Provide consistent DX and coverage across
all digital platforms
Support end user work flow across multi
screens
Consolidate resources & align business goals across platforms
Be ready for any new OS/Platform release
faster
5/26/18
5
Responsive Web Design (RWD) – Tools are already here
Responsive Web App – Test Plan Strategy: 6 Steps
5/26/18 10 © 2015, Perfecto Mobile Ltd. All Rights Reserved.
Choose The Right PlaZorms
Assure Visual Web Pages Across PlaZorms
Func3onality of Naviga3on Flows
and Menus
Client Side Performance
Tes3ng
Test Accessibility for Mobile and
Web
Test Environment Condi3ons (Networks, Sensors)
5/26/18
6
• Identify your object in a robust fashion that fits all digital Platforms
• Build object repository and use smart locators
Responsive Web Design (RWD) - Objects
The average website includes nearly 400 different objects. Now try loca3ng them on each and every DIGITAL plaZorm in your lab…
• Take screenshot and use Visual Checkpoint/assertion to validate responsive aspects
Responsive Web Design (RWD) – visual validation w/ Screenshots
5/26/18
7
Fundamentals of PWA Testing
© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Progressive Web App - Definition
Progressive Web Apps (PWAs) are web applications that are regular web pages or websites, but can appear to the user like traditional applications or native mobile applications. The application type attempts to combine features offered by most modern browsers with the benefits of a mobile experience (Wikipedia)
5/26/18 14 © 2015, Perfecto Mobile Ltd. All Rights Reserved.
5/26/18
8
The Rise of PWAs – Challenging Native Mobile Apps
A Progressive Web App is • Progressive -‐ Works for every user • Responsive -‐ Fits any form factor • Connec=vity independent • App-‐like -‐ Feels like an app • Safe -‐ Served via HTTPS • Search Friendly -‐allowing search engines to find it. • Re-‐engageable -‐Support push no3fica3ons. • Linkable -‐ does not require complex installa3on (store access)
5/26/18 15 © 2015, Perfecto Mobile Ltd. All Rights Reserved.
Source :medium.com
Progressive Web App – The Offline Network Benefit
5/26/18 16 © 2015, Perfecto Mobile Ltd. All Rights Reserved.
Source: VentureBeat
5/26/18
9
Progressive Web App - Architecture
5/26/18 17 © 2015, Perfecto Mobile Ltd. All Rights Reserved.
• A PWA App consists of few key components • Manifest.Json - That's the file within the PWA that describes the app, provides metadata specific
to the app like icons, splash screens, and more • Service Workers – A service worker is a script that your browser runs in the background, separate
from a web page, opening the door to features that don't need a web page or user interaction. Today, service workers already include features like push notifications and background sync Service worker push notification
PWA on iOS is Different than on Android
5/26/18 18 © 2015, Perfecto Mobile Ltd. All Rights Reserved.
iOS Android App type shortcut apk
Offline data Deleted aker idle 3me No limit
Offline storage (browser based) Safari < 50Mb Chrome 6% available storage
Sensors support Limited (BT) All
access to private informa3on No Yes
Payment system No supported Supported
Voice support No Yes
5/26/18
10
Progressive Web App – Test Plan Strategy: 6 Steps
5/26/18 19 © 2015, Perfecto Mobile Ltd. All Rights Reserved.
Validate PWA Manifest
Validate PWA Service Workers
Validate PWA Specific
Capabili3es
Properly Test Across PlaZorms
(RWD)
Object Iden3fica3on Strategy
Google’s PWA Checklist
Compliance
Progressive Web App – Test Plan Strategy: Step 5
5/26/18 20 © 2015, Perfecto Mobile Ltd. All Rights Reserved.
Object Iden3fica3on Strategy
&
5/26/18
11
Progressive Web App – Test Plan Strategy: Step 6
5/26/18 21 © 2015, Perfecto Mobile Ltd. All Rights Reserved.
Validate Your Tes=ng Plans Against Google’s PWA Checklist (Lighthouse) • https://developers.google.com/web/progressive-web-apps/checklist
Tool Selection Strategy
© 2015, Perfecto Mobile Ltd. All Rights Reserved.
5/26/18
12
How to select the right testing framework?
5/26/18 23 © 2015, Perfecto Mobile Ltd. All Rights Reserved.
Organiza=onal Fit Technical Fit
Selection Criteria: Organizational Fit Requirements
5/26/18 24 © 2016, Perfecto Mobile Ltd. All Rights Reserved.
Project Complexity
Quality analysis
requirements
Test Types & Prac=ces
Resources and Skills
Availability
Test Environment
Exis=ng Tech Stack
5/26/18
13
Selection Criteria: Technical Fit Requirements
5/26/18 25 © 2016, Perfecto Mobile Ltd. All Rights Reserved.
SDLC Process Fit
(Integra=ons, Plugins, etc.)
Community size, support and Doc’s
Feedback Loop and Repor=ng
Automa=on Coverage
Cloud and Automa=on at Scale
Automa=on Robustness and Maintainability
Web Testing Tools (Open-Source) - Download Trends
5/26/18 26 © 2016, Perfecto Mobile Ltd. All Rights Reserved.
5/26/18
14
Web Market Testing – Selection Criteria
5/26/18 27 © 2016, Perfecto Mobile Ltd. All Rights Reserved.
Read More Here
Headless Browsers Role
Motivation • Receive fast feedback (Unit and basic functional tests) – Mostly Dev • Easy environment setup, no IT dependencies • Uses standard JS language • Supported and promoted by browser vendors (Google, Mozilla) • Additional use cases to cover:
• HAR File • Performance testing • Basic UI • Security
5/26/18 28 © 2015, Perfecto Mobile Ltd. All Rights Reserved.
5/26/18
15
Grouping The Tools
5/26/18 29 © 2015, Perfecto Mobile Ltd. All Rights Reserved.
Real Browser (WebDriver) Headless
Headless browser SW (can used as standalone)
Used for: • End to end tes3ng • Func3onal tes3ng
Used for: • Fast CI – Short cycle • Basic func3onal tes3ng • Backend services tes3ng • Performance / network traffic
Tool Selection Case Study (USAA)
• Define needed capabilities
5/26/18 30 © 2015, Perfecto Mobile Ltd. All Rights Reserved.
Selec=on Criteria
End to End Tes3ng
BDD/ATDD Friendly
Tool Documenta3on
Visual Naviga3on Tes3ng
5/26/18
16
USAA Tool Selection
• Define needed capabilities • Identify importance (weight capabilities)
5/26/18 31 © 2015, Perfecto Mobile Ltd. All Rights Reserved.
Weight Key
1 – Low importance
3 – Medium importance
5 – High importance
Weight Selec=on Criteria 5 (High importance) End to End Tes3ng
3 (Medium importance) BDD/ATDD Friendly 5 (High importance) Tool Documenta3on 1 (Low importance) Visual Naviga3on Tes3ng
USAA Tool Selection
• Define needed capabilities • Identify importance (weight capabilities) • Define scoring key
5/26/18 32 © 2015, Perfecto Mobile Ltd. All Rights Reserved.
Scoring Key
0 – Did not meet expecta3ons
2 – Met expecta3ons
3 – Exceeded expecta3ons
Weight Selec=on Criteria Tool X Tool Y Tool Z 5 (High importance) End to End Tes3ng 3 3 3
3 (Medium importance) BDD/ATDD Friendly 3 2 3 5 (High importance) Tool Documenta3on 0 2 2 1 (Low importance) Visual Naviga3on Tes3ng 3 3 2
5/26/18
17
USAA Tool Selection
• Define needed capabilities • Identify importance (weight capabilities) • Define scoring key
5/26/18 33 © 2015, Perfecto Mobile Ltd. All Rights Reserved.
Weight Selec=on Criteria Tool X (weighted) Tool Y (weighted)
Tool Z (weighted)
5 (High importance) End to End Tes3ng 3 5 x 3 = 15 3 5 x 3 = 15 3 5 x 3 = 15 3 (Medium importance) BDD/ATDD Friendly 3 3 x 3 = 9 2 3 x 2 = 6 3 3 x 3 = 9
5 (High importance) Tool Documenta3on 0 5 x 0 = 0 2 5 x 2 = 10 2 5 x 2 = 10 1 (Low importance) Visual Naviga3on Tes3ng 3 1 x 3 = 3 3 1 x3 = 3 2 1 x 2 = 2
Total 27 34 36
Scoring Key
0 – Did not meet expecta3ons
2 – Met expecta3ons
3 – Exceeded expecta3ons
Infrastructure as a Freeway
5/26/18 34 © 2015, Perfecto Mobile Ltd. All Rights Reserved.
5/26/18
18
Thank You
top related