how to make test automation for cloud-based system
TRANSCRIPT
How to Make Test AutomationFor Cloud-Based System
by Nick Babich
Telephony Automation
TeamAlexey Georgy Alexandra
Nikita Alexey Igor2
Contents
1.RingCentral product and system specifics.
2.Automation: What-When-Where-How and Why
3.Questions and Answers3
4
Classic PBX System
5
Cloud Telephony System
RingCentral: Cloud Business Communications
6
RingCentral ServiceTelephony is a cornerstone service for the RingCentral. It
consist of many different features and each new release brings more to production.
7
8
Quality
Cost
Delivery
AutomationWhat?When?Where?How?Why?
9
What?Type of Automation 10
11
Testing Types
● Functional Tests● Acceptance
Tests
● A/B Tests
● Unit Tests ● Stress Tests● Security
Tests
Business Facing
Technology Facing
Supp
ort
Prog
ram
min
g
Criti
que
Pr
oduc
t
12
13
Most Common Use-Case
How to Define Scope for Automation● Automate the most critical cases (P0 cases,
especially one which are hard-to-execute manually).
● Automate the most frequently used scope (included in regular regression cycles).
● Automation for the most automatable scope (easy to automate and move to the execution).
Make a distinction between “ephemeral” and “long-standing” cases
14
Test Automation Criteria
15
When?General Automation
Flow16
Product Release Cycle
17
Agile Development
18
Testing Process
Create new test scope Define a scope for automation & automate it
Execute automated scope
Who: Scrum team QA engineers.
Task: Create test design for new features.
Deliverable: Prioritized test scope, proposal for automation
Who: Automation QA engineers.
Task: Automate test cases
Deliverable: Test scripts ready for execution
Who: Scrum team QA or DEV.
Task: Execute automated test cases.
Deliverable: Execution results. Understanding product quality.
19
Test Automation Plan
Development Team
Step 1: Starting development new telephony feature.
Development Team
Step 2: Prioritized test design for new functionality
Development Team
Step 3: Automation for the most critical test cases
Telephony Automatio
n Team
Step 4: Continue automation for 2-nd and 3-rd category
Telephony Automatio
n Team
Step 5: Delivering automated cases to the customers
20
Where?Test Environment21
22
Test Environment and Production
Test Environment Update
ADB
Codebase change or
Manual request
Start Automatic
Deployment System
SVNNew Build
23
Automatic Deployment System
3
2
1
Deploy → Verify (Post Deployment Verification) → Execute Regular Scope
24
25
Distributed Test Execution Scheme
How?Test Execution
Scheme26
Tools• Framework and automated tests cases (scripts).• Test Environment• Dashboard for test execution results and reports.• Bug tracking system
27
Technologies We Use
28
How It Works
29
Execution Scheme
Test Environment Deployment
Test Scope Execution
Send Execution Results to
Dashboar
d
New Build Available
30
Framework and Automated Test Cases
Business Layer
Test Scripts Layer
Application Part (exe)Command line application with a set of parameters (What we execute and Where)Set of modules required for test scripts (WebDriver, Rest API, Oracle ORA) Automated test cases (test scripts)
Powered by 31
Test Case Overview
32
Test design and Test script
33
Test script report
Powered by 34
• Start time (timestamp)• Group (e.g. feature)• Execution result (Passed / Failed / Skipped)• Screenshots and attachments (if test failed during execution ).
Execution Results for the Scope● Archive for all test reports
● Tested build (Branch and Build number)● Backend log data and system traces
Powered by 35
Execution Report in Dashboard• Configuration properties• Backend build (from execution results)• Execution Statistics (passed / failed / blocked)
36Powered by
Powered by
Dashboard Sample
37
Why?38
39
Metaphor Behind Test Automation
40
Cost: Manual vs. Automated Testing
41
Benefits of Automated Testing
Global Continuous Integration Team
6 Development Team
Regression Team
Total number of test cases: 5500Total number of Automated: 2100
On a daily basis
On a weekly basis
On a daily basis
Automated
Test Automation Disadvantages
42
1. Test scripts base requires regular support and actualization.
2. Troubleshooting for environment deployment and execution results analysis.
3. It works only on long-term basis.
Important Moments●Feedback from
stakeholders●Task Prioritization●Popularization of
automation 43
44
